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ETHICS  AUDITS:  EFFECTIVE  TOOLS  FOR  TRAINING  THE  ACQUISITION 

WORKFORCE 

Dr.  Robert  E.  Bateman, 

Weber  State  University 


ABSTRACT 

The  purpose  of  this  paper  is  to  focus  upon  the 
need  for  improved  awareness  and  sensitivity  to 
ethical  problems  and  issues  which  frequently 
confront  the  acquisition  workforce. 

The  proper  development  and  use  of  an  ethics 
audit  can  be  an  invaluable  tool  in  training  and 
orienting  the  acquisition  workforce  about  ethical 
problems  and  issues  which  often  confront 
acquisition  professionals.  The  ethics  audit  is  a 
practical,  flexible  and  imaginative  document  for 
assessing  and  introducing  ethical  problems  and 
issues  unique  to  an  organization,  whether  public 
or  private.  This  type  of  audit  is  a  systematic  and 
objective  survey  of  ethical  issues  and  provides 
yet  another  means  by  which,  awareness  of  and 
sensitivity  to  ethical  issues  can  be  introduced  in 
a  training  environment.  The  audit  most 
frequently  focuses  upon  the  following  key  areas 
(but  may  be  designed  to  introduce  and  explore 
many  other  appropriate  and  relevant  ethical 
issues): 

(a)  The  organization  and 
organizational  culture. 

(b)  Procedural,  policy  and  regulatory 

issues. 

(c)  Individual  behavior  and  conduct 

issues. 

(d)  Issues  unique  to  "the  system". 

Well  designed  audits  can  be  very  effective  tools 
in  identifying,  controlling  and  managing  ethical 
problems  within  any  organization.  Such  audits 


are  particularly  useful  in  providing  ethical 
education  for  both  new  trainees  and  experienced 
acquisition  personnel  within  the  spirit  and  intent 
of  the  Packard  Commission  Work  Force  Survey 
and  the  Defense  Management  Review  (DMR). 


INTRODUCTION 

One  of  the  prime  concerns  of  federal  agencies  in 
the  acquisition  process  is  the  maintenance  of  a 
high  level  of  organizational  and  professional 
ethics  and  integrity.  The  Federal  Acquisition 
Regulation  (FAR)  specifically  requires  that 
contractors  maintain  a  record  of  honesty  and 
integrity  to  assure  compliance  with  government 
regulations  and  thereby  assure  a  high  level  of 
moral  leadership  by  government  contractors. 
Ethical  problems  in  procurement  are  generally 
focused  into  two  general  areas.  The  first  area  is 
primarily  concerned  with  business  practices  and 
personal  conduct  of  individuals  that  is  improper. 
The  second  area  is  directed  toward  organizational 
conflicts  of  interest  arising  from  the  award  of 
contracts  to  individuals  and  organizations.  1 
Violations  of  procurement  ethics  can  be  very 
costly  to  government  contractors.  In  fact, 
violations  can  cost  a  company  the  contract  itself. 
A  company  could  lose  all  payments  received 
under  a  contact  and  could  ultimately  result  in 
criminal  fines  and  jail  time  for  corporate 
executives.  Government  contractors  should  take 
ethics  violations  very  seriously. 

In  June  1986  the  President's  Blue  Ribbon 
Commission  on  Defense  Management,  known  as 
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the  Packard  Commission,  issued  its  final  report 
entitled  A  Quest  for  Excellence.  2  The 
Commission  promulgated  under  Appendix  A, 
Conduct  and  Accountability,  what  was  known  as 
The  Defense  Industry  Initiatives  on  Business 
Ethics  and  Conduct,  which  established  certain 
guiding  principles  for  defense  contractors.  Under 
those  principles  contractors  were  to: 

1 .  have  and  adhere  to  written  codes  of  conduct; 

2.  train  their  employees  in  such  codes; 

3.  encourage  employees  to  report  violations  of 
such  codes,  without  fear  of  retribution; 

4.  monitor  compliance  with  laws  incident  to 
defense  procurement; 

5.  adopt  procedures  for  voluntary  disclosure  of 
violations  and  for  necessary  corrective  action; 

6.  share  with  other  firms  their  methods  for  and 
experience  in  implementing  such  principles, 
through  annual  participation  in  an  industry¬ 
wide  "Best  Practices  Forum";  and 

7.  have  outside  or  non-employee  members  of 
their  boards  of  directors  review  compliance. 

The  primary  focus  of  this  paper  is  on  the 
evaluation  and  assessment  of  ethical  conduct  and 
practices.  The  ethics  audit  should  be  considered 
and  utilized  as  one  of  several  acceptable  tools 
used  in  the  determination  of  ethical  conduct, 
practice  and  culture  within  the  acquisition 
workforce  and  even  beyond  to  the  entire 
acquisition  community  and  to  be  used  as  tools 
for  training  the  acquisition  workforce. 

The  Pentagon  scandal  of  1988,  known  by  many 
as  the  "Pentagate"  matter,  3  and  the  "Ill  Wind" 
probe  of  the  1 990 '  s  4,  which  disclosed  many 
problems  in  the  use  of  consultants  between  U.S. 
companies  and  the  Pentagon  and  raised  many 
ethical  issues  and  questions  about  competitive 
procurement  and  the  abuse  of  inside  information 
by  the  "revolving  door"  into,  out  of,  and  through 
the  aerospace  and  defense  industry.  In  view  of 
the  foregoing  considerations  one  might  naturally 
inquire  what  additional  steps  could  or  should 


Congress  and  the  Pentagon  take  to  manage  and 
control  ethical  conduct  in  the  aerospace  and 
defense  contract  industry?  It  is,  of  course,  most 
unlikely  and  improbable  that  such  ethical 
problems  will  be  totally  eliminated.  However, 
the  challenge  to  continually  train  the  workforce 
to  be  alert  to  ethical  issues  and  overcome  such 
problems  must  be  continuous  -  even  relentless. 

GOVERNMENT  PRINCIPLES  OF 
ETHICAL  CONDUCT 

Within  the  Federal  Government  continued 
emphasis  is  being  placed  on  ethical  issues  and 
problems.  For  example,  Executive  Order 
12674,  entitled,  "Principles  of  Ethics  Conduct 
for  Government  Officers  and  Employees",  was 
issued  on  April  12,  1988,  as  amended. 

Thereafter  the,  "Code  of  Ethics  for  Government 
Service",  was  issued  in  accordance  with  5 
U.S.C.A.  7301,  requiring  the  display  of  the 
"Code  of  Ethics  for  Government  Service"  in  any 
Federal  Government  building  in  which  at  least 
20  civilians  are  regularly  employed. 
Subsequently  under  the  provisions  of  Section  4, 
DOD  5500. 7-R,  the  Department  of  Defense 
established  "Department  of  Defense  Human 
Goals,"  and  Section  5  described  "Ethical  Values" 
considered  as  the  core  beliefs  which  are  to  be 
considered  and  used  by  DOD  employees  when 
making  ethical  decisions  as  part  of  their  official 
duties.  The  primary  ethical  values  identified  are: 
honesty,  integrity,  loyalty,  accountability, 
fairness,  caring,  respect,  promise  keeping, 
responsible  citizenship,  and  pursuit  of 
excellence.  Section  6  sets  forth  a  plan  for 
official  "Ethical  Decision  Making".  The  "ethical 
decision-making  plan"  includes  the  following: 
define  the  problem,  identify  the  goal(s),  list 
applicable  laws  or  regulations,  list  the  ethical 
values  at  stake,  name  all  the  stakeholders,  gather 
additional  information,  state  all  feasible 
solutions,  eliminate  unethical  options,  rank 
remaining  solutions,  and  commit  to  and 
implement  the  best  ethical  solution.  The 


foregoing  code,  values  and  plan  provide  a  clear 
and  comprehensive  expression  of  the  standards, 
principles,  processes  and  procedures  fundamental 
to  ethical  decisions  and  conduct  within  the 
Federal  Government  and  the  Department  of 
Defense.  Defense. 5 

MANAGING  AND  CONTROLLING 
ETHICAL  CONDUCT  WITHIN 
ORGANIZATIONS 

It  is  appropriate  to  note  that  there  is  significant 
professional  literature  and  documentation  to 
support  the  proposition  that  ethical  conduct  and 
practices  within  an  organization  can  indeed  be 
substantially  influenced  and  controlled.  6  A 
code  of  conduct  or  code  of  ethics  have  long  been 
proposed  and  utilized  as  an  important  aspect  of 
creating,  developing  and  controlling  the  conduct 
and  behavior  of  personnel  within  an  organization 
on  both  a  domestic  and  an  international  scale. 
Codes  of  conduct  are  considered  to  be  a  clear 
statement  of  an  organizations  core  values. 7 
Codes  of  conduct  or  ethics  are  considered  to  be 
the  core  or  dominant  values  and  beliefs  and  as 
such  are  often  considered  to  be  the  very 
foundation  of  the  corporate  culture.  For 
example  Caterpillar,  Inc.,  a  huge  multinational 
corporation,  issued  its  first  code  of  conduct 
which  was  entitled,  "A  CODE  OF 
WORLDWIDE  BUSINESS  CONDUCT  AND 
OPERATING  PRINCIPLES,  in  October,  1974. 
The  Chairman  of  the  Board  issued  the  following 
statement  concerning  the  publication.  "No 
document  issued  bv  Caterpillar  is  more  important 
than  this  one."  (underlining  supplied).  As 
Joseph  W.  Weiss  so  appropriately  observed, 
"Ethical  conduct  in  corporations  must  be  a 
shared  vision  and  practice  that  is  actively 
communicated  and  modeled  by  executives, 
managers,  and  employees  on  an  ongoing,  daily 
basis."  8  J.  L.  Brooks  offered  the  following 
recommendations  to  effectively  implement 
ethical  codes  of  conduct  into  an  organizational 
culture:  :(1)  confidentiality  should  not  be  so 


restrictive  that  it  prohibits  employees  from 
consulting  with  internal  personnel  regarding 
ethical  questions;  (2)  a  fair  and  objective  hearing 
process  should  follow  any  report  of  an  ethical 
problem  or  conflict;  (3)  assurance  should  be 
given  to  all  employees  that  their  rights  with 
respect  to  discrimination,  and  notice  and 
compensation  in  the  event  of  layoff  will  be 
protected;  (4)  A  section  on  employer  rights  or 
terms  of  employment  should  follow  the  employee 
rights  section,  in  order  to  facilitate  open 
communication  and  clarify  expectations;  (5) 
Executive  support  of  the  code  of  conduct  should 
be  demonstrated  in  the  form  of  clear  sanctions 
against  those  who  violate  it;  (6)  Conflicts  of 
interest  revolving  around  ownership  interest  in 
related  concerns  need  to  be  addressed,  as  these 
conflicts  can  cloud  judgments;(7)  Consider  any 
other  ethical  codes  that  influence  employees  in 
the  corporation,  and  design  the  corporate  code 
so  as  not  to  conflict  with  these  external  codes  of 
conduct.  "9 

ETHICS  AND  CONTROL  SYSTEMS 

Controls  and  control  systems  are  considered  to 
be  an  important  aspect  of  strategic  planning  and 
management.  In  a  broad  sense  strategic  planning 
and  management  is  vitally  important  to  the 
success  and  perpetuation  of  the  organization  to 
"control  the  control  system".  10  When  focusing 
on  the  need  for  an  effective  means  of 
establishing,  managing  and  enforcing  ethical 
conduct  within  an  organization  the  code  of  ethics 
is  considered  to  be  a  most  effective  tool.  Three 
important  aspects  of  controlling  organizational 
ethics  are  considered  to  be  organizational 
standards  and  codes  of  ethics,  measurement  of 
performance,  and  corrective  action.  1 1 

Considering  the  importance  of  ethics  initiatives 
within  the  Federal  Government  and  industry 
there  are  several  strategic  questions  that  should 
necessarily  be  posed.  Should  there  be  a  "control 
system"  i.e.  a  means  of  evaluating  the  system 
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regularly  or  on  a  periodic  basis?  The  author 
proffers  the  view  that  every  system  should  be 
reviewed  and  evaluated  regularly.  Is  the  system 
meeting  the  goals  and  objectives  for  which  it  was 
established?  Is  the  system  worth  the  commitment 
of  time  and  resources  expended?  If  not  why  not? 
If  not  what  is  required  or  necessary  to  return  it  to 
the  required  or  accepted  level  of  performance? 
What  are  the  internal  and  external  controls  of  the 
system?  Is  the  system  meeting  established  goals 
and  expectations?  Who  or  what  process  or 
system  controls  the  control  system? 

"Controlling  the  control  system  -  Control 
systems  operating  out  of  control  can  do  more 
damage  than  good.  Without  proper  care,  a 
variety  of  problems  can  develop."  12  It  is 
proposed  that  the  ethics  audit  provides  an 
effective  means  of  evaluating  the  ethics  "culture" 
or  "system"  within  an  organization. 

In  this  particular  regard  attention  should  be  given 
to  what  become  known  as  the  "COSO"  report. 
The  Committee  of  Sponsoring  Organizations  of 
the  Treadway  Commission  (COSO)  prepared  a 
report  entitled,  "INTERNAL  CONTROL  - 
INTEGRATED  FRAMEWORK"  in  September 
of  1992.  Oversight  of  and  representation  on  the 
Treadway  Commission  was  provided  by  the: 
American  Institute  of  Certified  Public 
Accountants,  American  Accounting  Association, 
The  Institute  of  Internal  Auditors,  Institute  of 
Management  Accountants,  and  the  Financial 
Executive  Institute.  The  Project  Advisory 
Council  of  COSO  consisted  of  representatives  of 
AT&T,  IBM  Corporation,  Shell  Oil  Company, 
Household  International,  The  University  of 
Arizona,  E.I.  Du  Pont  de  Nemours  and 
Company,  Nations  Bank  and  Arthur  Anderson  & 
Co.  The  author  of  the  report  was  Coopers  & 
Lybrand.13  The  report  defines  internal  control 
as  "a  process,  effected  by  an  entity's  board  of 
directors,  management  and  other  personnel, 
designed  to  provide  reasonable  assurance 
regarding  the  achievement  of  objectives  in  the 


following  categories:  1.  Effectiveness  and 
efficiency  of  operations,  2.  Reliability  of 
financial  reporting,  and  3.  Compliance  with 
applicable  laws  and  regulations".  Particularly 
emphasizes  was  the  fact  that  "everyone  in  an 
organization  has  responsibility  for  internal 
control."  14 

The  report  in  stressing  the  important  role  and 
responsibility  of  management  notes,  "More  than 
any  other  individual,  the  chief  executive  sets  the 
"tone  at  the  top"  that  affects  the  integrity  and 
ethics  and  other  factors  of  a  positive  control 
environment"  and  also  that  "all  personnel  should 
be  responsible  for  communicating  upward 
problems  in  operations,  with  the  code  of 
conduct,  or  other  policy  violations  or  illegal 
actions.  15  Among  the  factors  reported  to  be 
important  to  the  control  environment  was 
"integrity  and  ethical  values"  and  provides  that 
"Indications  of  lack  of  integrity  or  ethical  values 
in  any  endeavors  of  top  management— be  it 
executive,  operating  or  financial  management  - 
cast  a  pall  over  the  reliability  of  the  financial 
reporting  process. "  16 

The  "Evaluation  Tools"  portion  of  the  report  in 
noting  the  importance  of  integrity  and  ethical 
values  and  emphasized  that:  "management  must 
convey  the  message  that  integrity  and  ethical 
values  cannot  be  compromised,  and  employees 
must  receive  and  understand  that  message." 
The  following  six  points  of  focus  were 
emphasized  in  the  "Control  Environment" 
section  of  the  report: 

1 .  Existence  and  implementation  of  codes  of 
conduct  and  other  policies  regarding 
acceptable  business  practice,  conflicts  of 
interest,  or  expected  standards  of  ethical 
and  moral  behavior. 

2.  Establishment  of  the  "tone  at  the  top"  - 
including  explicit  moral  guidance  about 
what  is  right  and  wrong  -  and  extent  of 
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its  communication  throughout  the  organization. 

3.  Dealings  with  employees,  suppliers, 
customers,  investors,  creditors,  insurers, 
competitors,  and  auditors  etc.  (e.g. 
whether  management  conducts  business 
on  a  high  ethical  plane,  and  insists  that 
other  do  so,  pays  little  attention  to  ethical 
issues). 

4.  Appropriateness  of  remedial  action  taken 
in  response  to  departures  from  approved 
policies  and  procedures  or  violations  of 
the  code  of  conduct.  Extent  to  which 
remedial  action  is  communicated  or 
otherwise  becomes  known  throughout  the 
entity. 

5.  Management's  attitude  towards 
intervention  or  overriding  established 
controls. 

6.  Pressure  to  meet  unrealistic  performance 
targets  -  particularly  for  short-term 
results  -  and  extent  to  which 
compensation  is  based  on  achieving  those 
performance  targets.  17 

CODES  OF  ETHICS  - 
CODES  OF  CONDUCT 

Many  companies  attempt  to  provide  oversight 
and  governance  of  their  ethical  activities  through 
the  development  and  use  of  ethics  codes  or 
conduct  codes.  The  author  notes  that  references 
to  "codes  of  ethics"  and  "codes  of  conduct"  are 
often  used  interchangeably,  a  code  of  conduct  to 
one  organization  may  be  a  code  of  ethics  to 
another.  For  convenience,  and  to  avoid 
unnecessary  confusion  and  repetition,  future 
references  will  be  to  "  codes  of  ethics".  Codes 
of  ethics  have  been  identified  by  some  as  the 
means  by  which  an  organization  can  be  vitalized. 
18  It  has  been  observed  that  over  90  percent  of 
the  Fortune  500  companies  have  a  formal  ethics 


code.  19  A  survey  of  corporate  executives 
indicated  that  codes  of  ethics  are  the  most 
effective  way  to  encourage  ethical  corporate 
behavior.  20  Among  those  organizations  that 
have  developed  codes  of  ethics  the  ten  areas  of 
concern  often  included  are:  "1.  Exercise  of  due 
care  regarding  stakeholders.  2.  Confidentiality 
regarding  private  information.  3.  Fidelity  to 
special  responsibilities.  4.  Avoidance  of  the 
appearance  of  a  conflict  of  interest.  5.  Willing 
compliance  with  the  letter  and  spirit  of  the  law. 
6.  Acting  in  good  faith  in  negotiations.  7. 
Respect  for  human  well-being.  8.  Respect  for 
liberty  and  constitutional  rights  of  others.  9. 
Procedures  for  contesting  possible  code 
violations.  10.  Punishment  for  code  violations. " 
21  The  purpose  for  this  type  of  code  is  to 
establish  standards  and  to  identify  and  emphasize 
acceptable  codes  regarding  beliefs,  values  and 
conduct  and  as  such  set  a  moral  tone  for  the 
organization.  The  most  important  topics  often 
included  in  codes  of  ethics  include  "(1)  general 
statements  of  ethics  and  philosophy;  (2)  conflicts 
of  interest;  (3)  compliance  with  applicable  laws; 
(4)  political  contributions;  (5)  payments  to 
government  officials/political  parties;  (6)  inside 
information;  (7)  gifts,  favors,  entertainment;  (8) 
false  entries  in  books  and  records;  (9) 
customer /supplier  relations".  22 


LEADERSHIP  -  SETTING  THE 
"ETHICAL  TONE" 

It  is  important  to  note  that  the  "ethical  tone" 
within  an  organization  is  established  through  the 
actions  and  words  of  top  management.  23  The 
organizational  value  system  is  most  frequently 
established  by  the  top  executive  in  the 
organization.  24  Codes  of  Ethics  should  be 
published  and  "clearly  state  the  firms  core 
values."  25  Society  has  developed  a  higher 
level  of  expectation  and  recently  many 
organizations  have  made  substantial 
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commitments  of  time  and  capital  in  preparing 
rather  complete  and  strict  codes  of  ethics.  26  A 
number  of  multinational  corporations  have 
developed  worldwide  codes  of  ethics  which  they 
expect  to  be  followed  and  applied  by  all  of  their 
employees  throughout  the  world.  A  factor  or 
consideration  which  is  unique  to  this  type  of 
ethical  application  is  that  core  values  are  which 
are  emphasized.  Core  values  are  considered  to 
be  basic  and  so  fundamental  to  the  organization 
that  they  will  seldom  if  ever  be  compromised. 27 


STAKEHOLDERS 

Ethical  analysis  and  evaluation  of  an  organization 
must  identify  the  stakeholders  or  the 
constituencies  of  that  organization.  28 
Stakeholder  analysis  is  often  considered  to  be  a 
strategic  management  function  of  an 
organization.  29  A  stakeholder  is  "any  group 
that  has  an  actual  or  potential  interest  or  impact 
who  has  on  an  organizations  ability  to  achieve  its 
objectives. 30  Among  those  individuals, 
activities  and  associated  organizations  often 
considered  as  stakeholders  are: 
stockholders/owners,  customers,  employees, 
suppliers,  board  of  directors,  chief  executive 
officers  (CEOs)  and  other  top  executives,  media, 
consumers,  lobbyists,  courts,  governments, 
competitors,  the  public  and  society.  31  The 
stakeholder  concept  is  considered  to  be 
particularly  useful  when  considering  ethical 
analysis  because  "it  allows  managers  to  gauge 
the  impact  of  their  decision  on  several  affected 
groups."  32  Accordingly,  it  is  suggested  that 
any  in  depth  analysis  or  study  of  ethical 
problems  and  appropriate  methodologies  for 
improving  ethical  awareness  and  solutions  within 
an  organization  must  necessarily  consider  and 
carefully  identify  its  many  and  varied 
stakeholders  or  constituencies.  The  federal 
government  is  certainly  a  dynamic  stakeholder. 
Under  the  U.S.  Constitution  (Article  1,  Section 
8)  the  government  has  many  defined  roles  as 


"lawmaker,  regulator,  and  enforcer  over 
interstate  and  foreign  commerce,  taxes,  duties, 
imports,  tariffs,  excises,  the  military,  foreign 
affairs,  bankruptcy,  copyrights,  and  patent."  33 

ORGANIZATIONAL  CULTURE 

Organizational  culture  is  said  to  represent  "the 
sum  of  the  values  of  all  the  people  within  an 
organization.  It  is  the  normal  way  of  doing 
things  within  an  organization."  34 
Organizational  culture  has  also  been  defined  as 
"a  set  of  values,  beliefs,  goals,  norms,  and  ways 
to  solve  problems  that  members  (employees)  of 
an  organization  share.  "35  Indeed,  an  important 
component  of  organizational  culture  is  the  ethical 
climate  which  is  exists  within  the  organization  - 
the  organizational  conscience.  The  decision 
process  within  the  organization  determines 
whether  problems  are  considered  moral  or 
immoral.  36 

Factors  considered  to  be  important  to  the  ethical 
climate  include  codes  of  ethics,  actions  by  top 
management  regarding  ethical  issues,  ethics 
policies,  the  influence  of  coworkers,  and  the 
opportunities  for  unethical  conduct  and 
behavior. 37  The  organizational  culture  is  also 
considered  to  be  "the  shared  beliefs  top  managers 
within  a  company  have  about  how  they  should 
manage  themselves  and  other  employees,  and 
how  they  should  conduct  their  business."  38  To 
adequately  measure  and  assess  an  organizations 
cultural  values  a  culture  audit  may  be  performed 
to  identify  the  cultural  factors  within  the 
organization  that  significantly  contribute  to  and 
create  ethical  conflicts. 39 

The  author's  review  of  literature  on  business  and 
organizational  ethics  analysis  has  evoked  a 
conclusion  that  the  ethics  audit  of  an  organization 
should  be  but  one  means  of  evaluating  the 
organizational  culture.  An  organization  should 
necessarily  determine  the  extent  to  which  the 
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organization's  culture  contributes  to  the  ethical 
problems  which  are  normally  encountered.  Are 
systemic  ethical  problems  created  by  and 
inherent  to  the  culture  of  the  organization?  A 
more  comprehensive  cultural  evaluation  would, 
in  all  probability  require  a  cultural  appraisal  or 
audit. 

ETHICS  AUDITS 

Historically  audits  have  been  considered  as  a 
means  of  evaluating  the  health  or  condition  of 
the  affairs  of  the  an  organization.  The  nature  of 
auditing  has  been  defined  as  .  the 
accumulation  and  evaluation  of  evidence  about 
quantifiable  information  of  an  economic  entity  to 
determine  and  report  on  the  degree  of 
correspondence  between  the  information  and 
established  criteria.  .  ."40  Implicit  within  the 
foregoing  definition  is  the  fact  that  an  evaluation 
must  be  made  of  quantifiable  information  to  be 
measured  against  established  criteria  within  an 
organization  or  entity. 

The  ethics  audit  presupposes  that  ethical 
performance  should  be  reviewed  and  monitored 
to  insure  that  ethical  standards  and  expectations 
are  being  adopted  and  adhered  to  by  a  company, 
its  organizational  units  and  its  employees. 41 
The  ethics  audit  presupposes  a  measurement  or 
evaluation  of  the  ethical  condition  or  health  of 
the  organization  based  upon  established  ethical 
standards.  An  ethics  audit  should  be  conducted 
in  an  orderly  sequence.  It  should  be 
comprehensive,  systematic,  independent,  and 
periodic. 

THE  ETHICS  AUDIT 

Some  of  the  appropriate  questions  addressed  in 
ethics  audits  may  include: 

1 .  Who  are  the  formal  stakeholders? 

2.  Who  are  the  informal  stakeholders? 

3.  Who  are  the  primary  stake-holders? 


4.  What  are  the  control  systems? 

5.  Are  all  employees  provided  a  copy  of  the 
organization's  "code  of  ethics"or  "code 
of  conduct" . 

6.  Has  an  ethics  office  been  established? 

7.  Are  ethics  training  and  orientation 
programs  available  to  all  employees? 

8.  Are  all  employees  required  to  have  a 
basic  training  or  orientation  about  the 
organization's  ethics  program? 

9.  Are  current  policies  and  procedures  on 
ethics  available  to  all  employees? 

10.  Are  company  ethics  standards  and 
expectations  clearly  stated? 

11.  Is  there  a  company  ethics  office,  officer 
or  ombudsman? 

12.  Is  there  a  perceived  recognition  of  top 
level  executive  encouragement  and 
support  for  the  ethics  program  for  the 
organization? 

13.  Is  your  immediate  supervisor 
informed  of  the  company  ethics  policy 
and  program? 

14.  What  is  the  organization's  policy  and 
procedure  concerning  whistle  blowers? 

15.  Have  control  systems  for  the  ethics 
program  been  established? 

16.  Is  there  a  procedure  by  which  your  ethics 
question  can  be  heard  and  answered? 

17.  Is  there  a  procedure  by  which  your  ethics 
grievances  can  be  heard? 

18.  Is  there  a  procedure  by  which  your  ethics 
grievances  can  be  reviewed? 

19.  Is  there  a  procedure  by  which  your  ethics 
grievances  can  be  solved? 

20.  Is  there  an  ethics  peer  review  program? 

21.  What  is  the  companies  structure  for 
hearing  employee  complaints? 

22.  Are  ethical  policies  and  procedures  being 
enforced? 

23.  Is  there  a  policy  and  procedure  for 
identifying  and  resolving  conflicts  of 
interest? 

24.  Does  the  code  of  ethics  apply  equally  to 
all  employees,  managers,  and  executives? 
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25.  Is  the  code  of  ethics  updated  regularly? 

26.  Is  the  code  of  ethics  available  to 
everyone? 

27.  Has  an  ethical  checklist  been  prepared 
and  adopted  throughout  the  organization? 

28.  Does  the  organization  encourage 

questions  and  honest  inquiry? 

29.  Is  ethical  conduct  and  compliance  with 
the  code  of  ethics  truly  encouraged? 

30.  Does  the  organization  promote  and 

reward  ethical  conduct  and  practices.? 

31.  Does  the  code  of  ethics  address 

significant  managerial  issues? 

32.  Does  the  code  of  ethics  fairly  address  the 
needs  of  our  customers? 

33.  Does  the  code  of  ethics  accurately  and 
fairly  represent  the  needs  of  our 
employees? 

34.  Is  the  quality  of  our  products  and  services 
fairly  represented  to  the  public? 

35.  Is  the  organization  fair? 

36.  Does  the  organization  place  too  much 
emphasis  upon  profit? 

37.  Is  the  organization  truly  committed  to 
quality  service  and  excellent  value. 

38.  Are  you  proud  to  be  an  employee  and 
representative  of  this  organization? 

39.  Does  this  audit  ask  the  right  questions? 

40.  Do  you  believe  this  audit  equally  applies 
to  all  departments? 

41.  Is  this  audit  necessary  and  timely? 

42.  Are  there  other  or  more  important 
questions  that  should  be  asked  in  this 
audit? 

43.  Should  there  be  more  emphasis  on  the 
"bottom  up"  rather  than  the  "top  down" 
approach  to  addressing  ethical  problems? 

44.  Is  there  too  much  focus  on  stereotypes 
instead  of  diversity  and  cultural 
understanding? 

45.  Is  ethics  training  provided  and  are  ethical 
question  encouraged? 

The  sample  "business  ethics  audit"  provided  in 

EXHIBIT  1  ,as  presented  by  O.  C.  Ferrell  and 


John  Fraedrich,  provides  a  framework  for 
preparing  and  implementing  an  "ethics  audit" 
process  or  program  within  an  organization. 

ETHICAL  PERSPECTIVES  - 
PROPOSITIONS 

Dr.  Gene  R.  Laczniak  has  observed  that, 
"Proper  ethical  behavior  exists  above  the  law. 
The  law  merely  specifies  the  lowest  common 
denominator  of  acceptable  behavior.  "42  See 
EXHIBIT  2  Dr.  Laczniak's  "Ethical 
Propositions".  Similarly,  it  is  important  to 
understand  that  "ethics  is  not  simply  following 
the  law.  .  .  and  that  law  and  ethics  are  not 
identical. 43  Furthermore,  "...  all  laws  are  not 
necessarily  morally  defensible ".44  "Simply 
obeying  the  law  does  not  fulfill  all  ethical 
obligations  .  .  .  and  does  not  and  can  not  codify 
all  ethical  requirements ".45 

CONCLUSION 

"Organizations  desperately  need  managers  who 
can  accumulate  and  distribute  power  and  who 
can  solve  societal  conflicts  of  interest  in  a  fair 
and  equitable  manner.  Excellent  managers  are 
those  who  can  balance  or  integrate  the  sometimes 
conflicting  interests  of  owners,  employees, 
customers,  suppliers,  local  communities 
government  officials,  and  the  natural 
environment.  In  this  sense,  a  manager's  job  is 
more  similar  to  that  of  a  judge  and  legislator  than 
a  technician.  "46 

We  may  often  conclude  that  most  ethical 
problems  are  within  the  organization  and  may 
even  be  systemic  in  nature.  Solutions  to  ethical 
problems  are  usually  not  external  to  the 
organization  although  external  factors  definitely 
contribute  to  and  influence  ethical  problems. 
The  solution  to  such  problems  are  most  often 
internal  rather  than  external.  Admittedly,  one 
problem  is  based  on  how  internal  resources, 
attitudes,  policies  and  programs  are  used  in 
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responding  to  external  factors  or  influences. 
Ethical  analysis  and  problem  solving  is  a  never 
ending  process.  To  solve  the  ethical  problems  of 
an  organization  it  becomes  necessary  to  stand 
back  and  look  at  the  organization  and  its  unique 
culture.  Ethical  standards  and  codes  and 
organizational  compliance  and  response  should 
be  frequently  evaluated,  and  herein  proposed  - 
audited.  What  is  the  missions  and  goals  of  the 
organization?  Are  the  missions  and  goals  clearly 
defined  and  understood  by  the  top  managers  and 
all  employees?  Is  there  one  standard  that  applies 
to  all  who  are  associated  with  the  organization  or 
are  there  different  and  oftentimes  conflicting 
standards?  The  ethics  audit  can  be  an  effective 
tool  of  management  if  strategically  analyzed, 
utilized  and  applied. 

Ethical  growth  and  maturity  is  an  adaptive 
and  progressive  process  which  is  similarly 
important  to  the  growth  and  development  of 
individuals  and  organizations.  We  do  not 
suddenly  become  ethical  anymore  than  we 
suddenly  become  experienced  and  wise.  Our 
desire  and  commitment  to  be  just,  fair  and 
ethical  must  be  insatiable;  our  commitment 
must  be  deliberate  and  patient;  and,  our 
pursuit  must  be  focused  and  relentless.  As 
with  the  individual,  so  with  the  organization  - 
we  do  not  suddenly  arrive,  but  we  constantly 
become. 
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EXHIBIT  1 


Ethical  Propositions 

•  Ethical  conflicts  and  choices  are  inherent  in  business  decision  making. 

■  Proper  ethical  behavior  exists  on  a  plane  above  the  law.  The  law  merely  specifies 
the  lowest  common  denominator  of  acceptable  behavior. 

•  There  is  no  single  satisfactory  standard  of  ethical  action  agreeable  to  everyone 
that  a  manager  can  use  to  make  specific  operational  decisions. 

■  Managers  should  be  familiar  with  a  wide  variety  of  ethical  standards. 

•  The  discussion  of  business  cases  or  of  situations  having  ethical  implications  can 
make  managers  more  ethically  sensitive. 

•  There  are  diverse  and  sometimes  conflicting  determinants  of  ethical  action. 

These  stem  primarily  from  the  individual,  from  the  organization,  from  profes¬ 
sional  norms,  and  from  the  values  of  society. 

■  Individual  values  are  the  final  standard,  although  not  necessarily  the  determining 
reason,  for  ethical  behavior. 

■  Consensus  regarding  what  constitutes  proper  ethical  behavior  in  a  decision¬ 
making  situation  diminishes  as  the  level  of  analysis  proceeds  from  abstract  to 
specific. 

•  The  moral  tone  of  an  organization  is  set  by  top  management. 

•  The  lower  the  organizational  level  of  a  manager,  the  greater  the  perceived  pres¬ 
sure  to  act  unethically. 

■  Individual  managers  perceive  themselves  as  more  ethical  than  their  colleagues. 

•  Effective  codes  of  ethics  should  contain  meaningful  and  clearly  stated  provisions, 
along  with  enforced  sanctions  for  noncompliance. 

•  Employees  must  have  a  nonpunitive,  fail-safe  mechanism  for  reporting  ethical 
abuses  in  the  organization. 

•  Every  organization  should  appoint  a  top-level  manager  or  director  to  be  responsi¬ 
ble  for  acting  as  an  ethical  advocate  in  the  organization. 

Source:  From  Gene  Laczniak,  “Business  Ethics:  A  Manager’s  Primer, **  Business,  March  1983, 
pp.  23-29.  Copyright  1983  by  the  College  of  Business  Administration,  Georgia  State  Univer¬ 
sity,  Atlanta.  Reprinted  by  permission  of  the  publisher  and  author. 
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EXHIBIT  2 


Business  Ethics  Audit 


Organizational  Issues* 


YES 

NO 

1. 

Does  your  organization  include  ethical  concerns  in  strategic  planning  at  the  top  level  of 
the  organization? 

YES 

NO 

2. 

Does  top  management  have  a  mechanism  to  detect  ethical  issues  relating  to  employees, 
customers,  the  community,  and  society? 

YES 

NO 

3. 

Does  the  organization  communicate  information  about  trade  association  or  industry 
codes  of  ethics? 

YES 

NO 

4. 

Does  the  company  have  a  code  of  ethics? 

YES 

NO 

5. 

Is  there  formal  or  informal  communication  within  the  organization  about  procedures  and 
activities  that  are  considered  acceptable  ethical  behavior? 

YES 

NO 

6. 

Does  the  organization  communicate  its  ethical  standards  to  suppliers,  customers,  and 
significant  others  that  have  a  relationship  with  the  organization? 

YES 

NO 

7. 

Is  ethics  training  provided  for  top  management,  middle  management,  supervisory,  and 
other  employees  that  are  significant  in  the  operation  of  the  business? 

YES 

NO 

8. 

Do  the  company’s  manuals  and  written  documents  guiding  operations  contain  ethics 
messages  about  appropriate  behavior? 

YES 

NO 

9. 

Is  there  open  discussion  of  ethical  issues? 

YES 

NO 

10. 

Is  there  an  ethics  committee,  department,  team,  or  group  that  deals  with  ethical  issues  in 

the  organization? 

*A  high  number  of  yes  answers  indicates  ethical  mechanisms  and  procedures  in  place  within  the  organization. 


Examples  of  Specific  Issues  That  Could  Be  Monitored  in  an  Ethics  Audit** 


YES 

NO 

1. 

Are  systems  and  operational  procedures  for  individual  employees  to  safeguard  ethical 
behavior  absent? 

YES 

NO 

2. 

Is  it  necessary  for  employees  to  break  company  ethical  rules  in  order  to  get  the  job  done? 

YES 

NO 

3. 

Is  there  an  environment  of  deception,  repression,  and  cover-ups  concerning  events  that 
would  be  embarrassing  to  the  company? 

YES 

NO 

4. 

Are  participatory  management  practices  that  allow  the  discussion  of  ethical  issues  absent? 

YES 

NO 

5. 

Are  compensation  systems  totally  dependent  on  performance? 

YES 

NO 

6. 

Is  there  sexual  harassment? 

YES 

NO 

7. 

Is  there  any  form  of  discrimination — race,  sex,  or  age — in  hiring,  promotion,  or 
compensation? 

YES 

NO 

8. 

Are  the  only  concerns  about  environmental  impact  legally  required? 

YES 

NO 

9. 

Is  concern  for  the  ethical  value  systems  of  the  community  with  regard  to  the  firm’s 
activities  absent? 

YES 

NO 

10. 

Are  there  deceptive  and  misleading  messages  in  promotion? 

YES 

NO 

11. 

Are  products  described  in  a  misleading  manner  with  negative  impact  or  limitations 
uncommunicated  to  customers? 

YES 

NO 

12. 

Are  the  documents  and  copyrighted  materials  of  other  companies  used  in  unauthorized 
ways? 

YES 

NO 

13. 

Are  expense  accounts  inflated? 

YES 

NO 

14. 

Are  customers  overcharged? 

YES 

NO 

15. 

Is  there  unauthorized  copying  of  computer  software? 

**The  number  of  yes  answers  indicates  the  number  of  possible  ethical  issues  to  address. 

"Business  Ethics  Audit",  O.C.  Ferrell  and  John  Fraedrich, 
Business  Ethics:  Ethical  Decision  Making  Cases.  Second  Edition. 
Copyright  C  1994  by  Houghton  Mifflin  Company.  Reprinted  with 
permission. 
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ARMY  PROGRAM  MANAGERS:  A  COMPETENCY  PERSPECTIVE 


Captain  Bryan  J.  Me  Veigh,  U.S.  Army  Tank  Automotive  Command 
Dr.  Owen  C.  Gadeken,  Defense  Systems  Management  College 


ABSTRACT 

This  paper  summarizes  a  recently  completed 
Masters  Degree  research  project  at  the  Naval 
Postgraduate  School  (NPS)1.  The  project 
characterizes  the  competencies  which 
distinguish  the  most  successful  Army 
program  managers  (PMs)  from  their  peers. 
The  research  results  are  based  on  survey  data 
obtained  from  program  executive  officers 
(PEOs),  their  deputies,  program  managers, 
and  acquisition  students,  as  well  as 
interviews  conducted  with  selected  program 
managers.  Competency  rankings  from  the 
survey  are  compared  to  the  original 
competency  research  conducted  by  the 
Defense  Systems  Management  College 
(DSMC)  in  the  late  1980s.  The  study 
concludes  that  the  majority  of  the  original 
DSMC  competencies  are  still  valid  but  new 
competencies  also  emerged  which  may 
reflect  the  changing  program  management 
environment  of  the  1990s.  The  program 
management  curricula  of  DSMC  and  NPS 
were  also  examined  for  their  coverage  of  the 
most  important  competencies.  Although  the 
competencies  are  generally  addressed  by 
both  curricula,  the  study  recommends  an 
additional  block  of  instruction  on  “Marketing 
for  Program  Managers.”  This  block  would 
focus  on  how  successful  program  managers 
apply  the  most  important  competencies  in 
the  current  acquisition  environment. 

INTRODUCTION 

What  are  the  characteristics  that  distinguish 
the  most  successful  program  managers 
(PMs)  from  their  peers?  The  results  of  one 
of  the  most  extensive  studies  of  successful 


PMs  was  published  in  a  1989  article  entitled 
“The  Right  Stuff:  Results  of  DSMC’s 
Program  Manager  Competency  Study.”2 
The  ordinal  DSMC  study  identified  16 
competencies  in  its  PM  Job  Competency 
Model.  The  DSMC  study  defined  a 
competency  as  an  attribute  of  a  PM  that 
underlies  effective  performance.  The  DSMC 
PM  Job  Competency  Model  included  ten 
core  competencies  common  to  all  PMs,  and 
six  competencies  that  distinguished  the  most 
successful  PMs  from  their  contemporaries.3 
An  updated  version  of  the  DSMC 
competency  survey  was  developed  and 
administered  to  over  220  acquisition 
personnel  with  an  80%  response  rate.  The 
survey  asked  each  respondent  to  review  the 
27  competencies  from  the  original  DSMC 
survey.  They  were  then  asked  to  select  the 
nine  most  important  characteristics  of  the 
“ideal”  PM,  and  the  nine  least  important 
characteristics. 

Initially,  surveys  were  administered  to  Army 
PEOs  and  their  deputies.  These  senior 
acquisition  officials  were  asked  to  identify 
those  PMs  who  they  thought  best  fit  the 
competencies  that  they  had  previously 
identified  as  being  most  important  for  an 
“ideal”  PM.  They  selected  18  of  the  34 
Army  PMs  as  being  most  successful  and  16 
PMs  as  average.  [It  is  important  to 
understand  that  not  being  selected  as  one  of 
the  most  successful  PMs  did  not  mean  that 
the  remaining  PMs  were  poor  performers. 
For  the  purposes  of  this  study,  it  was 
assumed  that  anyone  selected  to  become  a 
PM  must  be  a  competent  (i.e.,  at  least  an 
average)  performer.]  Of  the  25  PMs  who 
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TABLE  I 


COMPARISON  OF  DSMC  AND  NPS  COMPETENCIES 


ORIGINAL  DSMC  COMPETENCY  MODEL 

REVISED  NPS  COMPETENCY  MODEL 

SENSE  OF  OWNERSHIP^) 

INNOVATIVENESS 

LONG-TERM  PERSPECTIVE 

POLITICAL  AWARENESS 

MANAGERIAL  ORIENTATION 

SENSE  OF  OWNERSHIP 

POLITICAL  AWARENESSH 

RELATIONSHIP  DEVELOPMENT  (*) 

optimizing:#) 

LONG  TERM  PERSPECTIVE 

RESULTS  ORIENTED 

COACHES  OTHERS(*)(@) 

SYSTEMIC  THINKING 

RESULTS  ORTENTED(*) 

INNOVATIVENESS 

MANAGERIAL  ORIENTATION 

FOCUS  ON  EXCELLENCE 

|  ACTION  ORIENTED 

ACTION  ORIENTED(*) 

!  PROACTIVE  INFORMATION  GATHERING 

RELATIONSHIP  DEVELOPMENT  (*) 

:  SELF  CONTROL  (*)(@) 

PROACTIVE  INFORMATION  GATHERING  ! 

SYSTEMIC  THINKING 

STRATEGIC  INFLUENCE  j 

FOCUS  ON  EXCELLENCE 

INTERPERSONAL  ASSESSMENT^) 

CRITICAL  INQUIRY 

CRITICAL  INQUIRY  I 

STRATEGIC  INFLUENCE 

AS  SERTIVENES  S  (#) 

INTERPERSONAL  ASSESSMENT^) 

NOTE: 


(*)  Indicates  a  competency  that  distinguishes  the  most  successful  PMs. 

(@)  Indicates  a  competency  that  was  added  in  the  revised  PM  Job  Competency  Model. 
(#)  Indicates  a  competency  that  was  deleted  in  the  revised  PM  Job  Competency  Model. 


Pentagon,  the  test  community,  contractors, 
and  the  user.  Many  PMs  stated  that  they 
were  able  to  obtain  positive  results  for  their 
programs  just  by  working  the  relationships 
that  they  previously  established  with  various 
agencies.  As  one  PM  stated:  This  may 
sound  like  the  “good-old-boy  network,  ” 
which  it  is,  but  the  Army  is  personnel  and 
staff  intensive,  so  interpersonal 
relationships,  the  ability  to  work  with  others, 
is  a  must. 

4.  Self  Control  (*):  Remaining  calm  and 
unemotional  in  stressful  situations. 

Several  of  the  PMs  that  were  interviewed 
said  that  they  knew  program  managers  who 
were  “screamers”  and  who  got  angry  very 
quickly.  While  this  may  work  in  the  short 


term,  PMs  who  resort  to  such  actions  will 
never  be  able  to  count  on  those  people  to 
“bend  over  backwards  for  them  in  the  long 
term.”  Most  PMs  agreed  that  there  is 
nothing  wrong  with  losing  your  temper  as 
long  as  you  understand  beforehand  the 
consequences  of  that  action. 

5.  Innovativeness:  Champions  and 
pushes  new  ways  of  meeting  program 
requirements. 

There  are  no  cookbook  answers  in  the 
acquisition  business:  if  there  were,  the  Army 
wouldn’t  need  PMs.  Program  managers  are 
hired  to  “manage  unique  situations 
effectively.”  It  is  important  for  a  PM  to 
remember  that  the  solution  that  worked 
yesterday  may  not  work  tomorrow.  The 
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responded  to  the  survey,  11  were  rated 
successful  and  14  were  average. 

REVISED  COMPETENCY  MODEL 

The  NPS  survey  validated  14  of  the  16 
competencies  identified  in  the  original 
DSMC  study.  A  comparison  of  the  original 
and  revised  Job  Competency  Models  is 
provided  in  Table  1.  Competencies  are  listed 
by  rank  order  of  importance  (from  survey 
data)  for  each  model. 

The  results  of  the  NPS  study  indicate  that  all 
PMs  share  11  competencies.  Additionally, 
there  are  five  competencies  which  appear  to 
distinguish  the  most  successful  PMs.  In 
order  to  gain  further  insight  into  these 
competency  rankings,  the  seven  most 
successful  PMs  (who  were  selected  by  more 
than  one  PEO)  were  interviewed.  The  16 
competencies  from  the  revised  Job 
Competency  Model,  their  definitions,  and 
insights  gained  from  these  interviews  are 
provided  below.  [Competencies  designated 
with  an  (*)  are  the  ones  that  distinguish  the 
most  successful  PMs  from  the  their  peers.] 

COMPETENCY  DEFINITIONS 

1.  Political  Awareness:  Knows  who 
influential  players  are,  what  they  want, 
and  how  best  to  work  with  them. 

Politics  affects  every  aspect  of  an  Army 
program.  Whether  this  is  right  or  wrong, 
PMs  must  operate  in  this  environment.  As 
one  PM  put  it:  If  you  are  not  politically 
astute  about  the  Pentagon,  Congress,  or 
other  Government  agencies,  you  will  never 
understand  their  agenda,  your  program  will 
fail,  and  you  will  never  know  why  it  failed. 

Program  managers  learn  very  quickly  how 
politically  sensitive  their  programs  are.  It  is 


important  for  PMs  to  understand  that  trade¬ 
offs  have  been  made  at  the  Army,  DoD,  and 
Congressional  levels  just  to  keep  their 
programs  viable.  As  one  PM  stated:  You 
may  have  to  concede  many  times  on  small 
issues  just  to  keep  your  program  alive;  lose 
the  battle  to  win  the  war. 

2.  Coaches  Others  (*):  Providing  others 
with  performance  feedback  and 
suggestions  to  improve  their  capabilities. 

As  the  Acquisition  Corps  evolves  into  a 
profession,  the  ability  of  junior  officers  to 
learn  from  their  superiors  will  help  them 
avoid  many  of  the  mistakes  of  the  past.  If 
PMs  do  not  train  the  people  who  will  one 
day  fill  their  positions,  they  are  doing  a 
disservice  to  the  Acquisition  Corps,  the 
taxpayer,  and  the  individual.  As  one  PM 
stated  about  one  of  his  junior  officers:  This 
guy  is  going  to  be  a  program  manager  one 
day;  it’s  my  job  to  coach  him  and  let  him 
develop  to  his  potential. 

3.  Developing  Relationships  (*):  Spends 
time  and  energy  getting  to  know  program 
sponsors,  users,  and  contractors. 

Program  managers  have  daily  interfaces  with 
a  variety  of  people  from  outside  their 
organizations.  Each  of  these  people  will 
have  their  own  agenda,  priorities,  and 
resource  constraints.  To  be  successful, 
program  managers  must  be  able  ...to pick  up 
a  phone,  explain  what  they  need  done,  and 
because  of  the  relationship  they  have 
established  with  these  folks,  have  them 
respond. 

Building  relationships  does  not  happen 
overnight.  PMs  must  take  the  time  to  visit 
and  cultivate  both  old  and  new  relationships 
with  key  personnel  affecting  their  programs. 
These  key  personnel  include  people  from  the 
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primary  responsibility  of  the  PM  is  to  find 
out  what  will  work,  and  do  it. 

6.  Results  Oriented  (*):  Evaluates 
performance  in  terms  of  accomplishing 
specific  goals  or  meeting  specific 
standards. 

One  of  the  most  important  parts  of  the  PM’s 
job  is  making  each  program  work  and 
getting  it  fielded.  As  one  PM  stated: 
Everything  you  do  [as  a  PM]  has  got  to  be 
focused  on  results,  results,  results.  To  get  a 
program  fielded,  PM’s  must  put  “marks  on 
the  wall;”  otherwise,  events  will  dictate  the 
course  of  the  program.  By  focusing  on 
results,  PMs  have  a  way  of  setting  priorities 
and  measuring  the  results  of  their  programs 
against  a  specific  standard. 

7.  Sense  of  Ownership/Mission:  Sees  self 
as  responsible  for  the  program; 
articulates  problems  or  issues  from  a 
broader  organizational  or  mission 
perspective. 

Sense  of  Ownership  is  important  from  the 
perspective  that  one  of  the  primary  roles  of 
PMs  is  to  be  their  program’s  number  one 
advocate  and  cheerleader.  One  PM  put  it 
this  way:  If  a  PM  is  not  out  there  leading 
the  charge  for  his  system,  then  he  is 
probably  doing  a  disservice  to  the  taxpayer 
and  the  soldier. 

A  negative  aspect  of  this  competency  is  that 
Sense  of  Ownership  might  imply,  to  some 
people,  that  the  program  belongs  exclusively 
to  the  program  manager.  PMs  should  never 
become  personally  attached  to  their 
programs.  They  must  keep  the  attitude  that 
“...if  the  Army  says  they  don’t  want  the 
program,  I  personally  don’t  want  it.”  This 
allows  them  to  establish  the  credibility  of 
their  programs  based  on  each  program’s 


capabilities,  without  others  viewing  their 
actions  as  personal  or  vindictive.  So  PMs 
must  be  committed  to  selling  the  Army’s 
program,  not  their  personal  program. 

8.  Long  Term  Perspective:  Anticipates 
and  plans  for  future  issues  and  problems. 

Most  PMs  agreed  that  the  most  commonly 
used  tool  for  focusing  on  the  long  term 
perspective  was  the  DoD  six  year  budget 
process.  One  PM  stated  that:  If  I  had  a 
[financial]  problem  today,  I  couldn  7  fix  it 
in  1994  if  my  life  depended  on  it,  without 
severe  turbulence,  because  I  don  7  have  the 
money  to  do  it.  The  1995  budget  is 
essentially  locked... it  would  take  a  miracle 
to  change  it  now.  So  1996  is  your  first  year 
to  impact.  Without  a  long  term  perspective, 
I  can  7  do  that. 

According  to  current  PMs,  what 
distinguishes  successful  PMs  from  their  peers 
is  their  ability  to  identify  a  crisis  that  will  not 
occur  until  two  years  from  now.  Successful 
program  managers  must  be  able  to  shape 
events  so  that  the  crisis  is  manageable  when 
it  does  occur. 

9.  Managerial  Orientation:  Gets  work 
done  through  the  efforts  of  others. 

Program  managers  don’t  have  time  to  do 
everything  themselves.  A  PM’s  job  is  to 
“steer  the  ship,”  keep  their  action  officers 
“heading  in  the  right  direction,”  and  then 
empowering  their  subordinates  to  complete 
the  mission. 

One  of  the  key  aspects  of  Managerial 
Orientation  is  that  it  forces  PMs  to  take  the 
time  to  understand  the  strengths  and 
weaknesses  of  their  personnel.  By 
understanding  subordinates’  capabilities,  the 
PMs  are  able  to  place  those  personnel  into 
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positions  that  maximize  their  strengths  and 
minimize  their  weaknesses.  By  empowering 
their  subordinates  to  accomplish  specific 
missions,  PMs  have  the  time  to  focus  their 
efforts  on  resolving  the  major  problems 
before  they  become  crises. 

10.  Action  Oriented:  Reacts  to  problems 
energetically  and  with  a  sense  of  urgency. 

A  program’s  performance  is  directly  tied  to 
how  action  oriented  its  PM  is.  To  quote  one 
program  manager:  You  don  7  need  me  if  I ’m 
not  action  oriented,  because  I’m  paid  to 
respond  to  a  crisis.  The  very  nature  of  the 
PM’s  job  means  that  they  must  deal  with 
crises  on  a  day-to-day  basis. 

Most  of  the  examples  cited  by  PMs 
concerning  this  competency  focused  on 
dealing  with  crises  which  pertained  to  the 
program’s  budget.  One  PM  put  it  this  way: 
When  the  Pentagon  calls  and  says  that  you 
have  two  hours  for  an  answer,  they  are  not 
throwing  "wolf  bait.  ”  You  have  got  two 
hours  to  get  them  an  answer  before  the 
window  of  opportunity  closes.  If  you  don  7 
respond,  you  have  just  lost  the  battle  - 
battles  which  normally  equate  to  money. 

11.  Focus  on  Excellence:  Strives  for  the 
highest  standards  regardless  of 
circumstance. 

Many  of  the  PMs  felt  that  it  was  easy  for 
people  to  say  that  they  focus  on  excellence, 
but  it  was  much  harder  to  actually  do  it.  As 
one  PM  stated:  I’ve  never  been  in  a 
situation  in  this  business  where  there  was  an 
excellent  solution.  Everything  is  a  trade-off 
So  PMs  strive  to  provide  the  user  and  the 
taxpayer  with  the  best  product  they  can 
within  the  constraints  of  cost,  schedule,  and 
performance. 


12.  Strategic  Influence:  Builds  coalitions 
and  orchestrates  situations  to  overcome 
obstacles  and  obtain  support. 

Strategic  Influence  plays  an  important  role  in 
the  external  environment  of  a  program.  It 
affects  how  a  program  is  funded,  staffed,  and 
fielded.  Program  Managers  can’t  get  their 
programs  fielded  by  themselves.  They  must 
be  able  to  build  coalitions  and  partnerships, 
and  be  able  to  effectively  use  them  to  weigh 
in  for  their  program  when  the  time  comes. 
One  PM  said  that  the  key  to  building  an 
effective  coalition  was  to:  ...come  across  as 
a  sincere  and  honest  person,  yet  willing  to 
stand  up  for  what  you  think  is  right,  then 
they  will  probably  be  willing  to  compromise 
if  you  don  7  have  a  dogmatic  approach  to 
things. 

13.  Critical  Inquiry:  Explores  critical 
issues  that  are  not  being  explicitly 
addressed  by  others. 

PMs  are  responsible  for  understanding  the 
political  environment  within  which  their 
programs  exists.  Their  skill  in  understanding 
other  people’s  agendas,  building  coalitions, 
and  getting  their  programs  fielded  depends 
on  their  ability  to  ask  the  hard  questions  “up 
front  and  early.”  It  is  this  Critical  Inquiry, 
asking  the  “what  if’  questions,  that  allows 
the  PM  to  discover  the  rationale  behind 
certain  answers.  As  is  often  the  case,  PMs 
can  gain  more  insight  from  understanding  the 
rationale  than  from  the  answer  itself. 

14.  Systemic  Thinking:  Organize  and 
analyze  problems  methodically. 

The  acquisition  of  a  major  weapon  system  is 
a  complex  process.  The  PM  must  not  only 
coordinate  the  program  through  its  current 
phase,  but  also  plan  for  the  program’s 
growth  throughout  its  life-cycle.  The  PM 
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must  be  able  to  methodically  lay  out  a  plan 
that  will  allow  the  program  to  get  through  a 
particular  event  as  well  as  future  events. 

15.  Proactive  Information  Gathering: 
Systematically  collects  and  reviews 
information. 

Very  few  people  are  willing  to  come  up  and 
tell  you  that  they  have  a  problem.  When 
something  goes  wrong,  people  will  tend  to 
“sit  on  the  news,  trying  to  make  it  better,  or 
hoping  the  bad  news  will  go  away.”  One  of 
the  PMs  put  it  this  way:  If  a  PM  is  not 
proactive,  he  cannot  get  his  job  done.  A  PM 
that  is  not  out  finding  his  problems  is  in  the 
reaction  mode.  If  I  am  reacting  to  a 
problem,  it  means  it ’s  already  here.  Already 
here  means  I  better  have  the  financial 
ability  to  do  it,  which  means  it ’s  too  late.  If 
I ’m  reacting  to  a  problem,  it  means  that  it ’s 
probably  already  over  my  head. 

In  short,  an  effective  PM  must  find  out  about 
problems  before  they  happen. 

16.  Interpersonal  Assessment  (*): 
Identifies  specific  interest,  motivations, 
strengths  and  weaknesses  of  others. 

Program  managers  must  be  cognizant  of 
their  own  strengths  and  weaknesses  as  well 
as  those  of  their  subordinates.  One  PM 
stated  that  the  reason  he  hired  a  specific 
deputy  was  to  balance  his  own  weaknesses: 
I  am  weak  in  program  management, 
budgeting,  cost  estimating  and  contract 
negotiating.  He  is  an  expert  in  those  areas. 
This  balances  my  own  style  of  management 
by  walking  around. 

By  understanding  their  subordinates’ 
capabilities,  PMs  will  be  better  able  to 
manage  their  programs.  They  will 
understand  when  to  get  out  of  their  way  and 


when  they  are  in  danger  of  “focusing  on  a 
few  trees  in  the  forest.”  In  short, 
interpersonal  assessment  is  the  ability  to 
understand  and  work  with  people. 

RECOMMENDATION 

Analysis  of  the  DSMC  and  NP5  curricula 
revealed  that  both  schools  provide  an 
awareness  of  the  1 1  core  competencies 
identified  in  this  research  project.  However, 
more  emphasis  is  needed  especially  on  the 
five  competencies  that  distinguish  the  most 
successful  program  managers.  Since  DSMC 
and  NPS  curricula  are  based  on  functional 
areas,  internal  curricula  changes  are  not 
recommended  since  they  may  alter  the 
functional  course  baselines  required  for 
Acquisition  Corps  Certification.  Instead, 
DSMC  and  NPS  should  develop  a  block  of 
instruction  specifically  devoted  to  the  set  of 
program  manager  competencies  and  how 
they  are  integrated  into  the  functions  PMs 
must  perform.  This  instruction  would 
include  real  examples  of  what  happens  to 
program  managers  who  understand,  or  fail  to 
understand,  particular  competencies. 

The  new  course  which  could  be  entitled 
“Marketing  for  Program  Managers”  could  be 
integrated  into  the  DSMC  and  NPS  curricula 
as  an  elective  or  seminar  program.  It  would 
help  ensure  that  future  program  managers 
are  provided  more  specific  guidance  on  how 
to  become  successful  in  their  dynamic  career 
field.  It  would  also  ensure  that  the  PM 
competencies  are  considered  as  educational 
programs  related  to  the  acquisition  process 
evolve  into  the  future. 

CONCLUSION 

The  primary  role  of  an  Army  PM  is  to  direct 
the  development  and  production  of  a  weapon 
system  within  the  constraints  of  cost. 
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schedule,  and  performance.  To  successfully 
accomplish  this,  PMs  must  exhibit  certain 
competencies.  Their  ability  to  integrate 
these  competencies  into  the  management  of 
their  programs  plays  an  important  part  in  the 
success  of  each  program.  The  revised  Job 
Competenciey  Model  identifies  16 
competencies  that  current  Army  major 
system  PMs  highlighted  as  being  important 
to  successful  program  management.  While 
these  16  competencies  can’t  be  seen  as  a 
“cook  book”  solution  for  future  PMs,  they 
do  provide  a  “blue  print”  of  the  leadership 
and  managerial  skills  needed  to  become  an 
effective  PM  in  today’s  turbulent  acquisition 
environment. 


BIBLIOGRAPHY 

1.  Me  Veigh,  Bryan  J.,  Army  Program 

Managers:  A  Competency  Perspective, 

Naval  Postgraduate  School,  Monterey, 
California ,  September  1994. 

2.  Gadeken,  O.  C.,  “The  Right  Stuff: 
Results  of  DSMC’s  Program  Manager 
Competency  Study,”  Program  Manager, 
September-October  1989. 

3.  Cullen,  B.J.,  and  Gadeken,  O.C.,  A 
Competency  Model  of  Program  Managers  in 
the  DoD  Acquisition  Process,  Defense 
Systems  Management  College,  Ft.  Belvoir, 
Virginia,  1990. 


23 


24 


Assessing  Acquisition  Management  Competencies  Using  Experiential  Classroom  Exercises 

Dr.  Owen  C.  Gadeken 
Defense  Systems  Management  College 


ABSTRACT 

The  passage  of  the  Defense  Acquisition 
Workforce  Improvement  Act  and  creation  of 
the  Defense  Acquisition  University  have 
resulted  in  major  changes  in  the  career 
development  and  training  of  the  Defense 
Acquisition  Workforce.  Hundreds  of 
competencies  (learning  objectives)  have  now 
been  identified  for  the  designated  acquisition 
career  fields.  Many  of  these  competencies 
go  beyond  the  knowledge  base  to  application 
and  synthesis  levels  of  Bloom’s  Taxonomy 
of  Learning.  This  large  number  of 
“performance-based”  learning  objectives  will 
not  only  require  new  classroom  teaching  and 
learning  approaches,  but  also  new 
approaches  to  assess  student  progress  and 
learning  outcomes.  This  paper  will  focus  on 
application  of  experiential  exercises  for  both 
learning  and  assessment  of  higher  level 
acquisition  management  competencies. 
Example  exercises  will  be  cited  from  team 
projects,  management  games  and  simulations 
currently  used  at  DSMC.  Through  such 
exercises,  students  learn  by  demonstrating 
both  their  acquisition  knowledge  and  their 
ability  to  act  on  this  knowledge  in  complex 
situations  which  mirror  those  they  will  face 
in  their  real  jobs. 

EXPERIENTIAL  EXERCISES 

An  experiential  classroom  exercise  is  an 
interactive  experience  emphasizing  learning 
by  doing,  and  providing  time  for  personal 
reflection  and  feedback  from  both  the 
instructor/facilitator  and  other  students. 
Frequently  these  exercises  involve  students 
in  a  variety  of  management  roles,  each 


related  to  key  issues  undergirding  the 
acquisition  scenario.  Each  role  contains 
extensive  information  on  past  activities, 
correspondence  on  current  practices, 
decision  situations,  and  critical  issues  faced 
in  the  scenario.  Through  observing  such 
behaviors  as  problem  solving,  decision 
making,  and  priority  setting,  students  are 
provided  feedback  on  their  behaviors,  and 
are  then  able  to  make  self-assessments  as  to 
the  effectiveness  of  such  behaviors. 

Experiential  exercises  provide  realistic  and 
focused,  but  neutral,  environments.  They 
allow  students  to  hone  in  on  both  their 
acquisition  knowledge  and  management  style 
as  strategic  and  tactical  issues  emerge-issues 
which  involve  coordination  and  integration 
among  individuals,  work  teams  and  outside 
stakeholders.  These  are  the  same  realities 
that  acquisition  managers  continue  to  deal 
with  as  shrinking  resources  and  revised 
objectives  force  them  to  face  the  paradox  of 
balancing  increasing  demands  with  increasing 
constraints  in  an  ever  changing  climate. 

Experiential  exercises  attempt  to  recreate 
real  world  acquisition  management  situations 
as  closely  as  possible.  The  goal  is  to  engage 
students  in  the  active  process  of  learning  by 
doing.  Using  multiple  roles  with  different 
information  and  perspectives,  these  exercises 
effectively  mirror  the  complexity  inherent  in 
real  life  experiences.  At  the  same  time, 
students  are  afforded  extensive  freedom  in 
the  behaviors  and  actions  they  choose  in 
response  to  scenario  issues.  This  insures  that 
follow-up  discussions  and  feedback  sessions 
relate  to  the  students'  actual  managerial 
styles. 
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The  major  learning  premise  underlying 
experiential  exercises  is  that  each  student  is 
responsible  for  his  or  her  own  learning 
coming  out  of  the  exercises.  Students  are 
therefore  encouraged  to  reflect  on  their  own 
judgment  and  actions  during  the  exercise, 
willingly  take  part  in  the  debriefing  sessions, 
and  give  and  receive  feedback. 

The  key  characteristics  of  experiential 
exercises  are  summarized  in  Figure  1.  The 
value  of  these  exercises  in  training  is  best 
expressed  by  Henry  Mintzberg  in  The  Nature 
of  Managerial  Work: 

Cognitive  study  is  useful  but  generally 
sterile.  Learning  is  most  effective 
when  the  student  actually  performs  the 
skill  in  as  realistic  a  situation  as 
possible  and  then  analyses  his 
performance  explicitly.  One  cannot 
learn  to  swim  by  reading  about  it.  One 
must  get  into  water,  splash  around, 
and  practice  various  techniques  with 
advice  from  someone  who  knows  what 
skills  swimming  requires.  Eventually, 
with  sufficient  feedback,  he  learns  to 
swim.  The  same  holds  true  for  many 
management  skills.  The  student  must 
be  immersed  in  the  milieu;  he  must 
practice  the  skill;  and  he  must  receive 
constructive  feedback  on  his 
performance  from  someone  who 
understands  the  skill. 

COMPARISONS 

This  section  offers  a  brief  comparison  of 
experiential  exercises  with  their  closest 
relatives  in  educational  methodology:  case 
studies  and  computer-based  simulations. 
Experiential  exercises  emphasize  learning 
through  doing,  as  compared  with  case 
studies  which  focus  on  the  application  of 
cognitive  learning  and  computer-based 


simulations  which  focus  on  analysis-based 
learning.  Case  studies  with  their  emphasis 
on  application  and  synthesis  in  the  cognitive 
domain  require  students  to  function  in  an 
almost  consultant-like  role.  Computer-based 
simulations  cause  students  to  compete  within 
the  constraints  of  a  programmed  model. 
Effectiveness  is  measure  against  the  model's 
design  rather  than  through  the  interactions 
and  behaviors  of  students.  The  similarities 
and  differences  between  these  learning 
vehicles  are  summarized  in  Figure  2. 

DSMC  EXAMPLES 

DSMC  uses  experiential  exercises 
throughout  its  curriculum.  Several  of  these 
simulations  are  listed  in  Figure  3.  They 
range  from  short,  narrowly-focused  activities 
to  complex,  elaborately-staged  exercises 
spanning  several  days.  Each  exercise  will  be 
briefly  discussed  in  this  section.  Further 
information  on  these  exercises  can  be 
obtained  from  the  DSMC  Education 
Department. 

Gold  of  the  Desert  Kings  is  a  half-day 
exercise  to  develop  goal  setting  and  team 
building  skills.  Students  form  teams  and 
compete  to  be  the  first  to  race  across  the 
desert,  mine  gold  in  the  mountains,  and 
return  safely  to  their  home  base.  DSMC 
licensed  this  exercise  from  Eagles  Flight,  a 
Canadian  firm  which  specializes  in 
developing  short  experiential  exercises. 

The  next  two  experiential  exercises  were 
developed  in  conjunction  with  the 
Management  Simulation  Projects  Group  at 
New  York  University.  The  first  exercise 
focuses  on  key  issues  involved  in  contract 
management  and  is  set  in  a  government 
contracting  office.  This  exercise  is  used  as 
an  instructional  tool  in  a  two-week 
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acquisition  management  course  for  senior 
contracting  personnel. 

The  second  exercise,  Foodcorp  II,  is 
designed  to  deal  with  executive  skills  in  the 
context  of  operating  a  major  food  company. 
This  exercise  focuses  primarily  on  observing 
management  and  leadership  behaviors  using 
Baldrige  and  Excelsior  Quality  Management 
Award  Criteria  as  a  baseline.  Foodcorp  is 
offered  and  as  featured  parts  of  DSMC's  14- 
week  Advanced  Program  Management 
Course  and  3 -week  Executive  Management 
Course. 

Looking  Glass  was  the  first  behavioral 
exercise  used  by  DSMC.  It  was  developed 
by  the  Center  for  Creative  Leadership  in 
Greensboro,  North  Carolina.  The 
commercial  manufacturing  scenario  is  not 
closely  related  to  that  found  in  Defense 
organizations.  However,  the  scenario's 
managerial  issues  have  considerable  parallel 
to  those  faced  by  Defense  managers.  The 
commercial  scenario  also  provides  a  "neutral 
playing  field"  for  Defense  managers  to  focus 
on  developing  their  leadership  and 
management  skills  (instead  of  concentrating 
on  content  issues  in  the  scenario). 

DSMC's  newest  experiential  exercise,  Moon 
Base  Alpha,  was  taken  from  real  U.S. 
National  Aeronautics  and  Space 
Administration  (NASA)  project  files. 
Students  become  program  managers  in  an 
international  space  agency  charged  with 
developing  a  lunar  mining  base.  Through  a 
series  of  integrated  vignettes,  students  are 
required  to  demonstrate  key  program 
management  skills. 

The  Mouse  Trap  Car  is  an  adaptation  of  the 
popular  university  engineering  demonstration 
project.  Students  form  contractor  teams  and 
work  through  the  Defense  acquisition 


process  to  develop  prototype  model  cars. 
The  exercise  culminates  with  a  competitive 
run-off  in  front  of  the  entire  student  body. 

Grand  Slam  is  the  most  elaborate  of  DSMC's 
experiential  exercises.  It  covers  the  entire 
life  cycle  of  a  Defense  missile  program  with 
the  class  acting  as  the  program  office. 

ASSESSMENT  EXAMPLES 

A  key  challenge  in  using  experiential 
exercises  in  Defense  acquisition  career 
training  is  how  to  assess  required  student 
competencies.  Much  of  the  early 
development  of  experiential  exercises 
concentrated  on  feedback  and  self- 
assessment  of  students’  managerial  and 
leadership  behaviors.  However,  recent  work 
at  DSMC  has  been  focused  on  adapting 
experiential  exercises  for  more  rigorous 
assessment  of  designated  acquisition  career 
competencies.  The  remainder  of  this  section 
will  highlight  assessment  techniques  now 
used  in  DSMC’s  Moon  Base  Alpha  exercise. 

The  Moon  Base  Alpha  scenario  puts 
students  into  seven  interrelated  program 
managers  role  in  an  international  space 
agency  (ISA)  which  is  developing  a  lunar 
base.  The  ISA  organization  chart  is 
illustrated  in  Figure  4.  Moon  Base  Alpha 
consists  of  three  sequential  exercises,  an 
initial  in-basket  followed  by  two  team 
performance  exercises.  Learning  objectives 
run  the  spectrum  from  acquisition  knowledge 
to  leadership  skills  as  shown  in  Figure  5. 

The  in-basket  exercise  can  be  assessed  in  a 
variety  of  ways  depending  on  the 
competencies  being  assessed.  Leadership 
and  management  competencies  can  be  assess 
using  “model”  in-baskets  which  detail 
expected  responses  for  self-scoring  by 
students.  An  example  in-basket  issue  and 
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“model”  response  are  illustrated  in  Figures  6 
and  7.  A  more  rigorous  assessment  of 
students’  acquisition  functional  competencies 
is  illustrated  in  Figure  8.  After  reading  the 
in-basket  materials,  students  respond  to  a 
number  of  proposed  program  actions  in  each 
functional  area.  The  responses  are  again 
scored  against  a  model  as  illustrated  in 
Figure  9.  In  fact,  responses  on  this  type  of 
assessment  can  be  scored  simultaneously 
against  different  types  of  competencies  as 
shown  in  Figure  10. 

Team  performance  exercises  can  also  be 
scored  using  a  variety  of  techniques.  One  of 
these,  the  peer  rating  sheet,  is  illustrated  in 
Figure  1 1 . 
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SUMMARY 

The  benefits  of  using  experiential  exercises  in 
training  are  summarized  in  Figure  12. 
Student  feedback  has  supported  the  premise 
that  such  exercises  are  powerful  learning 
tools  that  can  concentrate  over  a  short 
period  of  time  what  can  take  much  longer  to 
learn  whether  on  the  job  or  in  a  more 
traditional  classroom  environment. 

Currently,  acquisition  organizations  are 
competing  in  increasingly  uncertain, 
changing,  risk-filled  times.  A  great  premium 
will  be  paid  for  developing  top  flight 
acquisition  managers  who  can  operate 
effectively  in  this  constantly  changing  and 
highly  constrained  resource  environment. 
Just  as  this  new  environment  calls  for 
acquisition  managers  to  develop  new  and 
more  effective  management  approaches,  so 
too  will  the  organizations  which  train  these 
future  managers  need  to  develop  new  and 
more  effective  training  methods  exemplified 
by  experiential  exercises. 
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Figure  1.  CHARACTERISTICS  OF  EXPERIENTIAL  EXERCISES 
o  Realistic  situation 
o  Complexity  of  issues  and  opportunities 
o  Multiple  roles  with  different  information  and  perspectives 
o  Experiential  learning  ("learning  by  doing") 
o  Freedom  of  action  by  students 
o  Extensive  feedback  on  student  behaviors  and  results 
o  Emphasis  on  reflection  and  self-assessment 


Figure  2.  COMPARISON  OF  EDUCATIONAL  METHODOLOGIES 


Materials 

Case  Studies 

One  case 

Computer  Simulations 

One  case  with  computer 
instructions 

Experiential  Exercises 
Individually 
tailored  cases 

Viewpoint 

Students  view  case 
from  same  perspective 
(as  external  consultants) 

Students  view  case  from 
same  perspective  (as 
competitors) 

Each  student  has  a 
different  role, 
information  and 
perspective 

Approach 

Analyze  problems  and 
propose  solutions 

Students  compete  with 
each  other  and  against 
computer  model 

Students  assume 
roles  with  instructors 
as  observers/role 
players 

Class  Session 

Instructor-led  seminar 

Interaction  with  computer 

Interactive  experience 

Results/Product 

Logical  analysis  and 
acceptable  solutions 

Computer  analysis  and 
comparison  with  other 
students 

Personal  feedback  on 
both  content  (what 
was  done)  and 
process  (how  it  was 
done) 

Primary 

External  assessment 

External  assessment 

Self  assessment 
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Figure  3.  DSMC  EXPERIENTIAL  EXERCISES 


Simulation 

Length 

Gold  of  the  Desert  Kings 

1/2  day 

Contract  Simulation 

1  day 

Foodcorp  II 

1  day 

Looking  Glass 

2-3  days 

Moon  Base  Alpha 

2  days 

Mouse  Trap  Car 

2  weeks 

Grand  Slam 

2  weeks 

Purpose 

Goal  Setting/Team  Building 

Contract  Management 

Strategic  Planning/Quality  Management 

Management/Executive  Skills 

Program  Management 

Program  Management 

Program  Management 


Figure  4.  ISA  ORGANIZATION  CHART 
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Figure  5.  MOON  BASE  ALPHA  LEARNING  OBJECTIVES 


•  Technical  Competencies 


•Management  Competencies 


-Acquisition  Strategy 
-Test  and  Evaluation 
-Ensuring  Design  Meets  Requirements 
-Cost/Schedule  Performance 
-Contract  Management 


-Relationship  Development 
-Managerial  Orientation 
-Political  Awareness 
-Political  Awareness 
-Strategic  Influence 


•  Essential  PM  Elements 


•Time  Management  Elements 


■Cost 

■Schedule 

■Performance 

■Supportability 


-Process 

-Perception 

-Prioritization 

-Judgment 


Figure  6.  MOON  BASE  ALPHA  IN-BASKET  ISSUE 

ISSUE  SUMMARY:  A  relatively  small  company,  Glenn-Tech  Inc.,  beat  an  industry  giant, 
Prentice  Industries,  for  the  waste  management  system  contract.  Although  the  contract  is  well 
underway,  Prentice  has  filed  a  formal  retroactive  complaint.  The  basis  of  their  complaint  is  that 
Glenn-Tech  has  failed  to  use  the  key  personnel  or  facilities  which  they  originally  proposed  on  the 
waste  management  system  contract.  It  appears  that  Prentice  carries  a  great  deal  of  weight  and  is 
trying  to  have  a  Stop  Work  Order  issued  while  their  claim  is  investigated. 

Questions  for  Discussion 

1 .  Per  the  ISA  General  Counsel,  the  PM  will  be  required  to  present  a  rebuttal  to  the  formal 
complaint.  What  information  will  be  needed  (e.g.,  source  selection  process  &  criteria)  and 
who  and  where  will  the  rebuttal  information  be  derived  from? 

2.  How  can  Glenn-Tech’ s  excellent  performance  to  date  be  used  as  an  argument  to  negate  the 
complaint  and  issuance  of  a  Stop  Work  Order? 


3.  What  contingency  plans  are  necessary  in  the  unfortunate  event  that  a  Stop  Work  Order  is 
issued? 
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Figure  7.  MOON  BASE  ALPHA  MODEL  ISSUE  RESPONSE 


ISSUE;  FORMAL  COMPLAINT  OVER  THE  AWARD  OF  THE  WASTE 
MANAGEMENT  SYSTEM  CONTRACT 


SUGGESTED  ACTIONS: 

RELATIONSHIP  DEVELOPMENT  (RD) 

1.  Contact  Shaqueena  Walker,  General  Counsel,  to  introduce  yourself  and  to  mention  that  your 
staff  is  preparing  data  which  will  document  the  contractual  fairness  o  the  waste  management 
system  selection  process  (Actively  introduce  self  to  new  personnel). 

2.  Inform  the  PEO  of  the  complaint  (Make  superiors  aware  of  potential  problems). 
MANAGERIAL  ORIENTATION  (MO) 

3.  Task  the  Chief  of  Procurement  to  brief  you  and  compile  critical  contract  selection  data  (as 
referenced  in  Item  16  model  responses)  (Delegate  tasks  to  appropriate  personnel). 

STRATEGIC  INFLUENCE  (SI) 

4.  Discuss  this  problem  with  the  Acquisition  Executive  (Owens)  at  Friday’s  staff  meeting  (Item 
26).  Inquire  as  to  which  persons  will  make  up  the  General  Counsel.  Emphasize  to  Owens  that 
Glenn-Tech  (current  contractor)  is  doing  an  excellent  job,  and  ask  Owens  for  recommendations 
and  his  support  on  stopping  the  possible  Stop  Work  Order  (Cultivate  support  of  powerful 
person). 

SUMMARY  OF  ACTIONS  TAKEN: 


RD 

1 

Perceived/Performed: 

Did  not  Perceive/Perform: 

RD 

I 

Perceived/Performed : 

Did  not  Perceive/Perform: 

MO 

3 

Perceived/Performed : 

Did  not  Perceive/Perform: 

SI 

4 

Perceived/Performed: 

Did  not  Perceive/Perform: 
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Figure  8.  MOON  BASE  ALPHA  DECISION  RANKING  ASSESSMENT  EXAMPLE 
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Figure  11.  MOON  BASE  ALPHA  EXAMPLE  PEER  RATING  SHEET 
Ratee:  PM  -  Large  Work  Facilities  (LWF) 


Overall  Competency  Rating  Behavior  Rating 

5  -  extremely  effective  X  -  strength 

4  -  more  than  effective  0  -  needs  improvement 

3  -  effective  N  -  not  observed/neutral 

2  -  marginally  effective 
1  -  not  effective 


Strongest  Competency  Competency  Most  in  Need  of  Improvement 

Relationship  Development  X  _ 

Results  Orientation  ________  ___________ 

Assertiveness  ___________  X 

Optimizing  _^_____  _______ 
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Figure  12.  BENEFITS  OF  EXPERIENTIAL  EXERCISES 


o  Are  inherently  interesting  and  motivating 

o  Allow  tailored  assignments  to  fit  student  needs 

o  Allow  students  to  experience  the  different  stakeholder  perspectives  that 
affect  the  outcome  of  major  program  decisions  and  actions 

o  Give  students  a  feeling  (or  gestalt)  of  what  it  is  like  to  manage  in  a 
complex,  realistic  program  management  environment 

o  Put  the  focus  squarely  on  the  leadership  and  management  skills  found  to 
be  most  critical  to  success 

o  Provide  real-time  feedback  from  multiple  sources  (self,  peers  and  facilitators 
as  well  as  comparison  with  prior  exercises) 

o  Encourage  transition  from  classroom  learning  to  job  application 
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ACQUISITION  PLANNING 
AND  MANAGEMENT 
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COMPARATIVE  TRENDS  IN  ONGOING  ACQUISITION  RESEARCH 


James  W.  Abellera 
Defense  Systems  Management  College 


ABSTRACT 

Previous  research  has  shown  that  the  aims 
of  acquisition  studies  conducted  within 
universities  differ  from  those  of  acquisition 
studies  conducted  by  or  for  the  DoD.  This 
study  compares  recent  shifts  in  the 
distribution  of  broad  topics  for  the  two 
research  communities .  The  results  show 
that  acquisition-research  priorities  for  both 
groups  changed  during  the  1993-94  period. 
Although  their  priorities  did  not  converge , 
the  communities  moved  in  generally 
parallel  research  directions. 


BACKGROUND  AND  THE  PROBLEM. 

In  1989  the  Defense  Systems  Management 
College  started  a  research  program  called 
"Research  on  Ongoing  Acquisition  Research 
(ROAR)"  to  discover  the  types  and  extent  of 
ongoing  defense-related  acquisition  policy 
studies  [1]  conducted  by  professionals  inside 
and  outside  the  DoD.  The  purpose  of 
ROAR  was  to  create  an  information  system 
to  satisfy  two  objectives. 

The  first  objective  was  to  enable 
acquisition  researchers  to  access  ROAR 
information  freely  and  discover  other 
researchers  who  were  working  on  similar 
acquisition  issues.  This  would  help  them  to 
accelerate  their  studies  through  mutually 
benefical  exchanges  of  empirical  data  and 
experiences. 

Second,  DoD  policymakers  could  use 


ROAR  information  to  review  ongoing 
acquisition  studies  on  topics  of  immediate 
policy  interest.  Thus  they  might  avert 
spending  money  for  new  research  starts  that 
would  duplicate  work  already  in  progress. 

Or  policymakers  might  quickly  obtain  useful 
findings  from  projects  that  were  nearing 
completion  but  were  not  yet  published  or 
otherwise  identifiable  through  a  search  of 
existing  databases,  such  as  the  Defense 
Technical  Information  Center  (DTIC), 
Defense  Logistics  Studies  Information 
Exchange  (DLSIE),  CompuServe,  and 
traditional  library  indexes. 

As  the  program  began,  ROAR 
researchers  agreed  that  the  program  would 
capture  abstracts  of  ongoing  research  from 
all  major  professional  sources.  It  was  also 
understood  that  the  DoD  finances  the 
production  of  thousands  of  reports  related  to 
acquisition  policy  each  year.  The  research 
underlying  the  reports  is  conducted  by  the 
DoD’s  own  employees,  by  the  employees  of 
its  Federally  Funded  Research  and 
Development  Centers  (FFRDCs),  and  by 
for-profit  contractors.  [2] 

Centralized  DoD  bibliographic 
sources  (used  by  DoD  researchers  and 
policymakers  to  discover  published  and 
ongoing  studies  in  defense  acquisition 
policy)  gave  ample,  albeit  incomplete, 
evidence  of  research  performed  within  and 
funded  by  the  DoD.  Within-DoD  research 
is  defined  here  to  include  studies  performed 
by  the  DoD’s  employees,  for-profit 
contractors,  and  FFRDCs.  The  same 
centralized  sources  offered  little  evidence 
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that  research  was  performed  outside  the 
Defense  Department  —  by  professors  and 
fellows  in  universities  for  example.  [3] 

However,  an  informal  review  at  that 
time  of  articles  in  well-known  professional 
journals  for  selected  disciplines  (e.g., 
engineering  management,  economics,  law, 
political  science,  operations  research, 
finance,  business  management,  etc.) 
suggested  that  university  scholars  were 
conducting  defense-related  acquisition  policy 
studies  as  well. 

To  ensure  ROAR’s  coverage  of 
acquisition  studies  would  not  overlook  any 
major  source  of  such  research,  the  program 
gathered  information  about  ongoing  research 
projects  at  universities  and  other  outside- 
DoD  organizations  (e.g.,  non-profits  without 
affiliation  with  the  DoD  and  trade 
associations)  as  vigorously  as  it  collected 
information  on  within-DoD  efforts. 

Abstract  data  collected  by  ROAR 
from  1989  to  1992  showed  that  42%  (370  of 
876)  of  ongoing  acquisition  studies  were 
conducted  within  universities.  Also,  a  study 
of  funding  sources  for  the  370  university- 
based  research  revealed  that  the  Defense 
Department  financed  only  5%  of  such 
studies.  The  remaining  95%  of  university 
projects  were  funded  primarily  by  the 
universities  themselves.  Private  foundations 
and  corporations  also  furnished  support.  [4] 

The  1989-92  data  revealed  that 
university  projects  differed  from  within- 
DoD  studies  in  the  distribution  of  broad 
acquisition  topics  each  community  studied. 
Compared  to  within-DoD  projects,  the 
university  studies  were  skewed  toward 
issues  concerning  Commercial  Products  & 
Practices,  Defense  Acquisition  &  the 
Political  Process,  Engineering  & 
Manufacturing  Matters,  International 


Acquisition  Issues,  and  the  Industrial  Base. 

Compared  with  within-DoD  projects, 
the  university  work  was  skewed  away  from 
issues  concerning  Cost  &  Price 
Considerations,  Decision  &  Data  Support, 
Financial  Management,  Streamlining 
Acquisition  Management.  [5] 

HYPOTHESIS 

It  was  expected  that  the  priorities  of 
both  the  within-DoD  and  the  university 
research  communities  would  shift  with  the 
passage  of  time.  One  theory  about  the 
direction  of  changes,  which  could  be 
explored  in  the  distribution  of  topics  for 
ongoing  studies  captured  by  ROAR  after 
1992,  was  that  each  community  would  move 
roughly  in  tandem  with  the  other.  That  is, 
a  sharp  change  in  one  community’s 
emphasis  toward  or  away  from  one  major 
topical  area  —  relative  to  the  pattern  of 
1989-92  —  would  also  occur  in  the  same 
direction  for  the  other  community. 

The  rationale  for  tandem 
distributional  changes  over  time  is  that 
researchers  in  both  communities  attempt  to 
formulate  studies  that  are  responsive  to 
current  issues  and  problems  facing 
acquisition  policymakers.  While  their 
funding  sources  and  backgrounds  may  be 
different,  both  types  of  researchers  want 
their  studies  to  answer  existing  or 
foreseeable  public-policy  needs. 

Such  needs  are  transmitted  in  new 
federal  programs,  in  national  policy 
statements  by  the  President  and  other 
leaders  in  the  executive  branch,  and  by 
pronouncements  from  members  of  congress 
and  from  industry  leaders. 
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Of  course,  no  relationship  may  exist 
between  priority  shifts  of  each  community. 
In  this  case  one  expects  to  see  a  roughly 
even,  and  therefore  random,  mix  of  tandem 
and  non-tandem  trends. 


In  this  study,  ROAR  abstract  data 
collected  from  1993  through  1994  are 
analyzed  and  compared  with  similar  data  for 
the  1989-92  period  to  determine  if  within- 
DoD  and  university  researchers  generally 
changed  their  research  priorities  in  tandem. 


To  simplify  tandem  analysis,  the 
analysis  focuses  on  topical  areas  showing 
shifts  greater  than  five  percentage  points 
relative  to  the  1989-92  baseline. 


[  This  column  is  intentionally  blank  ] 


FINDINGS  AND  DISCUSSION 

Table  1.  below  shows  percentage 
distributions  by  broad  research  topic  for  the 
within-DoD  and  university  research 
communities  combined ,  for  the  1989-92 
period  and  for  the  1993-94  period.  The 
latter  period  includes  1,332  studies  that 
were  captured  by  the  ROAR  program  after 
31  December  1992.  Data  for  the  former 
period  were  presented  in  a  1993  study  as 
simple  frequency  distributions.  [6] 


Topical  areas  that  show  shifts  greater 
than  five  percentage  points  across  the  two 
periods  are  underlined. 


In  the  1993-94  sample,  42%  of  the 
1,332  newly-captured  ongoing  studies  were 
conducted  by  university  researchers. 
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Table  1.  Topical  Area-Shifts  in  ROAR  Studies  Across  Periods 


Topical  Area 


Incidence  Incidence 
89-92  (n=876)  93-94  (n=l332)  Difference 


Commercial  Products  &  Practices 

Engineering  & 

Manufacturing  Matters 

Contracting  & 

Subcontracting  Issues 

Industrial  Base 

Cost  &  Price  Considerations 

Streamlining  Acquisition  Mgmt 

Defense  Acquisition 
&  the  Political  Process 

Acquisition  Management 
Education  &  Workforce 

International  Acquisition  Issues 

Financial  Management 

Logistics  Issues 

Decision  &  Data  Support 


Percentages  displayed  in  each  of  the 
first  two  columns  of  numbers  do  not  sum  to 
100%  because  most  studies  were  classified 
as  belonging  to  at  least  two  topical  areas 
simultaneously.  Some  studies  were  assigned 
to  three  or  even  four  topical  areas  because 
of  the  nature  of  the  specific  research. 

Six  topical  areas  underlined  in  Table 
1 .  indicate  shifts  exceeding  five  percentage 
points  across  the  periods. 


13% 

38% 

25% 

14% 

26% 

12% 

24% 

34% 

10% 

22% 

33% 

10% 

21% 

27% 

6% 

16% 

17% 

1% 

12% 

12% 

0% 

18% 

18% 

0% 

10% 

9% 

-1% 

10% 

8% 

-2% 

26% 

22% 

-4% 

31% 

20% 

-11% 

Data  for  six  groups  is  further 
disaggregated  below,  in  Table  2.,  to  show 
shifts  that  occurred  for  the  within-DoD  and 
the  university  communities  across  the  two 
periods. 
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Table  2 .  Comparative  Topical-Area  Shifts  by  Communities 


Topical  Area 

Commercial  Products 
&  Practices 

Engineering  & 
Manufacturing  Matters 

Contracting  & 
Subcontracting  Issues 


Industrial  Base 

Cost  &  Price 
Considerations 

Decision  &  Data  Support 


Within-DoD 
Community  (W) 


89-92 

93-94 

10% 

22% 

12% 

21% 

24% 

46% 

21% 

30% 

25% 

31% 

39% 

25% 

University 
Community  (U) 


89-92  ! 

93-94 

16%  | 

62% 

o\° 

CO 

H 

34% 

25% 

19% 

25% 

36% 

16% 

22% 

19% 

13% 

Period-Shift 


Differences 
(Wf  (U) 

12% 

46% 

9% 

16% 

22% 

-6% 

9% 

11% 

6% 

6% 

-14% 

-6% 

Table  2.  data  confirm  that,  for  five 
of  the  six  topical  areas  in  which  sharp  shifts 
occurred  in  the  incidence  of  ongoing 
studies,  the  trends  occurred  in  tandem  (in 
the  same  direction)  between  the  within-DoD 
and  the  university  research  communities. 
However,  the  small  number  of  data  points 
does  not  make  statistical  testing  useful  for 
rejecting  or  accepting  the  counter  view  that 
the  observed  outcomes  are  simply  random 
occurrences. 

With  regard  to  convergence, 
inspection  of  the  frequency  distributions  for 
the  two  communities  indicates  that  they 
were  not  congruent  in  1989-92,  nor  did  they 
become  so  in  1993-94. 

While  evaluation  of  all  of  the 
comparative  sub-trends  suggested  by  the 
data  is  beyond  the  scope  of  the  present 
study,  the  single  instance  of  non-tandem 


trends,  in  the  Contracting  &  Subcontracting 
Issues  area,  warrants  a  word  of  discussion. 

The  recent  high  DoD  priority  of 
reforming  the  acquisition  process  may 
explain  why  the  incidence  of  contracting- 
related  studies  for  the  within-DoD 
community  jumped  22  percentage  points 
from  1989-92  to  1993-94. 

During  the  later  period,  however, 
defense  contracting’s  specialized  nature  may 
have  made  useful  studies  difficult  for 
university-based  researchers,  because 
thousands  of  rules  were  subject  to 
substantial  and  sudden  changes.  Thus,  the 
small  decline  in  incidence  for  universities  of 
six  percentage  points  could  reflect  the 
relative  disadvantage  perceived  by  college 
researchers  in  making  valuable  contributions 
during  a  time  of  rapid  internal  DoD  changes 
to  these  complex  rules. 
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CONCLUSIONS 

Analysis  of  ROAR  abstract  data 
spanning  six  years  does  not  disconfirm  the 
theory  that  both  the  within-DoD  and  the 
university  communities  of  acquisition 
researchers  shifted  their  study  priorities  in 
tandem.  One  of  six  areas  in  which  sharp 
shifts  occurred  offers  contrary  evidence. 

There  is  no  evidence  in  the  presented 
data  that  the  acquisition-research  priorities 
of  the  two  communities  converged  after 
1992,  nor  is  there  evidence  that  convergence 
is  more  or  less  likely  than  other  possible 
outcomes  in  future  years. 

It  does  appear  that  university 
researchers  responded  in  1993-94  to 
emerging  acquisition  issues  in  roughly  the 
same  way  as  their  within-DoD  counterparts. 
The  evidence  is  not  strong  however. 

Whether  tandem  shifts  in  priorities  continue 
will  depend  partly  on  how  well  each 
community  tries  to  meet  the  demand  for 
effective  public  policies  with  timely 
acquisition-related  research  products. 

FOOTNOTES 

1.  Defense  acquisition  policies  delineate 
between  acceptable  and  unacceptable 
management  practices  for  controlling  how 
the  DoD  acquires  goods  and  services. 
Defense  acquisition  policy  research  is 
defined  here  as  research  that  improves 
understanding  about  effective  practices  at 
various  levels  of  control  and  also  about 
internal  and  external  circumstances  that  can 
cause  the  DoD  to  change  its  policies.  Some 
policies  describe  practices  that  are  unique  to 
DoD.  Others  deal  with  practices  that  are 
commonly  used  in  less-tightly  regulated 
organizations  such  as  companies  and  non- 
DoD  government  agencies. 


2.  Part  of  the  annual  publications  output  is 
reported  in  Defense  Logistics  Studies 
Information  Exchange,  1993  Annual 
Department  of  Defense  Bibliography  of 
Logistics  Studies  and  Related  Documents. 
Department  of  Defense,  1993,  pp.  1-663. 
The  volume  lists  over  1400  titles  produced 
by  or  for  DoD  each  year.  Although  less 
than  half  of  the  logistics-related  reports 
listed  are  considered  by  the  author  to 
involve  acquisition  policy  research,  logistics 
concerns  form  only  one  of  many  areas 
germane  to  acquisition  policy. 

3.  Ibid.,  pp.  664-661.  For  example,  23 
logistics-related  projects  were  performed  by 
universities  out  of  2,805  total  projects 
completed  and  published  with  DoD  funding 
during  a  two-year  period.  Also,  the  IDA 
(Institute  for  Defense  Analyses)  Cost 
Research  Symposium  began  an  innovative 
program  in  the  early  1980s  to  identify  and 
disseminate  news  about  ongoing  research  on 
cost-estimating  models  and  other  cost  issues 
in  DoD  acquisition.  In  1989  IDA  reported 
209  projects  to  the  Symposium’s  within- 
DoD  participants.  No  university  research 
was  included;  all  reported  projects  in  1989 
were  performed  by  within-DoD 
organizations.  See  S .J.  Balut  and  K.L. 
Wilson,  "The  IDA  Cost  Research 
Symposium,"  Institute  for  Defense 
Analyses,  IDA  Document  D-647,  August 
1989,  various  pages. 

4.  Abellera,  J.  W. ,  "National  Resources 
for  Defense  Research,"  Proceedings,  1993 
Acquisition  Research  Symposium,  Defense 
Systems  Management  College  and  the 
Washington,  DC  Chapter  National  Contract 
Management  Assoc.,  pp.  3-4. 

5.  Ibid.,  pp.  5-7. 

6.  Ibid. 
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REDESIGNING  THE  DEFENSE  ACQUISITION  PROCESS 

by 

Henry  C.  Alberts,  Professor  of  Acquisition  Management 
Defense  Systems  Management  College 


ABSTRACT 

Over  a  period  of  six  years,  the  Technical 
Managers'  Advanced  Workshops  (TMAW) 
studied  the  Defense  acquisition  process.  The 
first  workshops  sought  to  define  what 
Program  Managers  did  and  the  kinds  of 
things  that  affected  their  work.  The  second 
set  of  workshops  was  sponsored  by  the  then 
incumbent  Under  Secretaries  of  Defense 
(Acquisition)  to  examine  in  detail  the 
problems  experienced  by  Program  Managers 
which  caused  program  execution  to  differ 
from  program  plan.  Other  workshops 
sponsored  or  requested  by  Program  Offices, 
the  Defense  Inspector  General's  Office 
(DoDIG),  the  Defense  Intelligence  Agency 
(DIA)  and  the  Office  of  the  Secretary  of 
Defense  (OSD)  examined  specific  kinds  of 
difficulties  experienced  within  the  acquisition 
process.  Finally,  another  series  of  workshops 
determined  the  functions  performed  in 
creating  effective,  supportable  weapon 
systems  and  defined  an  acquisition  process 
around  those  functions.  That  accomplished,  a 
workshop  was  held  to  see  if  implementing  the 
newly  designed  functional  process  would 
alleviate  the  difficulties  previously 
experienced  with  the  Defense  Acquisition 
System  (DAS). 

This  paper  provides  a  broad  overview  of  that 
work. 

THE  WORKSHOPS  THAT  PROVIDED 
ACQUISITION  DESIGN  KNOWLEDGE 


dates  of  the  workshops;  (2)  workshop 
sponsor;  (3)  general  area  of  investigation,  and 
(4)  where  applicable,  the  focus  questions 
posed  to  workshop  participants.  The 
workshop  results  and  the  methodology  used 
to  achieve  those  results  were  reported  in 
DSMC  publications  entitled,  "Technical 
Managers'  Advanced  Workshop  (TMAW) 
Reports".  TMAW  reports  were  not  widely 
distributed.  Much  information  was  elicited 
under  DSMC's  " Non-Attribution "  rule  and 
the  sources  of  the  data  are  protected.  For 
workshops  sponsored  directly  by  the  Under 
Secretary  of  Defense  (Acquisition)  USD(A) 
or  by  the  Director  of  Defense  Research  and 
Engineering  (DDR&E),  special  reports  were 
issued  under  their  imprimatur  and  distributed 
by  their  office.  In  the  case  of  Concurrent 
Engineering  workshops,  a  report  and  annexes 
were  widely  distributed  throughout  the  DoD, 
the  individual  Services,  and  the  defense 
industrial  base.  During  the  past  three  years, 
concurrent  engineering  has  become 
institutionalized  throughout  the  defense 
community  as  Integrated  Product 
Development  (IPD)  and  Integrated  Product 
and  Production  Teams  (IPPT). 

Each  workshop  consisted  of  between  eight 
and  14  participants  drawn  from  all  of  the 
stakeholder  organizations.  Participants 
represented  a  full  spectrum  of  perceptions  and 
perspectives  of  most  individuals  involved 
with  the  DAS.  In  all,  over  300  individuals, 
each  selected  by  their  Commands  because  of 
specific  knowledge  of  the  workshop  subject 
matter,  contributed  to  the  redesign  process. 


Twenty  eight  workshops  were  held  during  a 
39  month  period.  Table  1  provides:  (1)  the 
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Table  1 

TMAW  WORKSHOPS 


WORKSHOP  GROUPS  AND  DATES 


Group  1  DSMC  Sponsored 
06/15  -  06/19  1988 
11/30-  12/04  1988 
05/26  -  05/30  1989 


Group  2  USD(A)  Sponsored 

08/11  through  10/13  1989 
08/11  to  10/13 
09/12  to  09/13 
09/19  to  09/21 
10/11  to  10/13 
12/04  through  12/08  1989 


Group  3  DoBIG  Sponsored 
10/02  to  10/04  1989 
11/27  to  11/29  1989 

Group  4  USD(A)  Sponsored 
03/14  to  03/15  1990 
07/17  to  07/19  1990 
11/06  to  11/07  1990 
11/26  to  11/27  1990 
11/29  to  11/30  1990 
12/11  to  12/12  1990 
08/19  to  08/21  1991 


Group  5  OSD  Elements  Sponsored 
02/12  to  02/14  1989 
06/25  to  06/29  1990 
12/17  to  12/21  1990 
01/22  to  01/26  1991 
01/28  to  01/29  1991 
02/27  to  03/02  1991 
05/22  to  05/24  1991 
08/28  to  08/30  1991 


Group  6  DSMC  Sponsored 
06/21  to  06/25  1991 
07/23  to  07/24  1991 


WORKSHOP  SUBJECT  MATTER 


"What  Do  Technical  Managers  Do?" 

"What  Forces  Cause  Change?" 

"How  should  Technical  Managers  Handle  Change?" 

"Why  Do  Smart  Munitions  Cost  More  and  Why  Are  They 
Late? 

Air  to  Surface  Programs 
Surface  to  Surface  Programs 
Surface  to  Air  Programs  and  ASW  Programs 
Air  to  Air  Programs  and  ASW  Programs 
"Suppose  Smart  Munitions  Changes  Were  Made?" 


"Performing  the  Test  &  Evaluation  Inspection" 
"Should  HARPOON  Become  Multi-Source?" 


"Define  and  Discuss  Program  Stability" 

"What  Problems  Are  There  In  Reducing  Risk" 
"Problems  of  Aligning  Requirements  and  Resources 
"Why  are  Firms  Leaving  the  Industrial  Base?" 

"What  Are  The  Problems  With  Program  Oversight?" 
"How  to  Improve  The  Industrial  Base>  " 

"The  Future  Role  Of  The  Industrial  Base" 


"System  Engineering  Round-Table"  DSMC 
"Problems  With  Implementing  CALS"  OSD  CALS 
"Implementing  Concurrent  Engineering"  OSD 
"Developing  CITIS"  DDR&E,  CALS 
"Concurrent  Engineering  Planning  Session"  OSD 
"Joint  U.S.  Canada  Technology  Integration"  ASD(PR) 
"Which  Technology  Should  Be  Integrated?"  ASD(PR) 
"Contractor  Integrated  Technical  Information"  OSD 


"Integrating  An  Acquisition  Process  Data  Base" 
"Integrating  An  Acquisition  Process  Data  Base" 


WHAT  THE  WORK  SHOPS  REVEALED 

The  first  group  of  workshops  examined  the 
functions  performed  by  Defense  Program  and 
Technical  Managers.  Workshop  participants 
described  the  function  in  general  terms  and 
when  asked  what  problems  they  had  in 
carrying  out  their  responsibilities,  responded 
by  stating  their  frustration  with  the  lack  of 


program  stability,  and  their  own  inability  to 
satisfy  the  need  to  remain  flexible; 
specifically,  to  waive  use  of  inappropriate 
specifications  and  regulations  when  that  was 
necessary  to  maintain  program  integrity. 

The  second  group  of  workshops  was  held  for 
the  USD(A)  who  invited  all  major  smart 
munitions  program  managers  and  their 
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contractor  counterparts  to  participate.  The 
workshop  recommendations  focused  on 
change  to  the  acquisition  management 
structure  in  order  to  give  program  managers 
authority  commensurate  with  their 
responsibilities.  The  last  workshop  in  this 
series  prepared  recommended  changes  for 
USD(A)  implementation.  While  considering 
those  recommendations,  it  became  clear  that  a 
methodology  which  could  predict  how  the 
DAS  would  respond  to  proposed  change  was 
lacking.  Unfortunately,  the  workshop  held  to 
discuss  that  subject  could  not  find  a  more 
rigorous  approach  than  what  had  been 
employed  in  the  past. 

The  third  group  of  workshops  explored  the 
DoD  Inspector  General  role  in  two  specific 
instances:  (1)  How  the  DoDIG  might  best 
structure  an  inspection  into  DoD  Test  and 
Evaluation  procedures;  and  (2)  When  it 
would  be  appropriate  to  change  acquisition 
strategy  from  prime  contractor  to  "break-out" 
of  major  portions  of  the  system  for  direct 
Program  Office  management.  Participants 
described  difficulties  in  holding  to  promised 
schedule  and  performance  objectives  when 
financial  and  methodological  reviews  were 
initiated  for  established  on-going  programs. 

The  fourth  group  of  workshops  was  directed 
by  USD(A)  to  assist  in  developing 
improvement  to  the  DAS.  The  participants 
were  selected  by  USD(A)  and  were  asked  to 
consider  all  of  the  problems  they  had 
experienced  with  a  view  toward  generating 
actions  USD(A)  could  take  to  help  prevent 
those  problems  from  recurring.  The 
workshops  reinforced  the  need,  stated  in  the 
first  three  workshops,  for  flexibility  in 
dealing  with  change.  In  fact,  workshop 
participants  described  program  stability  as, 
"the  ability  to  deal  with  change  in  a  timely 
manner".  Participants  also  found  that  a  great 
deal  of  risk  could  be  introduced  by  resource 


uncertainties,  and  that  the  difficulty  in 
properly  aligning  resources  with  program 
requirements  was  due  to  inflexibility  of  the 
DAS  process;  and  that  some  firms  were 
leaving  the  defense  industrial  base  because  of 
the  imposition  of  oversight  requirements 
unnecessary  and  harmful  to  the  conduct  of 
normal  business  activities. 

The  fifth  group  of  workshops  dealt  with 
technology  integration  and  change  in  program 
management  methodologies  made  possible 
through  improved  technical  equipment  and 
concepts.  Focus  was  on  the  search  for  ways 
to  encourage  progress  toward  a  paperless 
society,  and  with  integration  of  various  skills 
within  program  offices  to  achieve  integrated, 
multi-skilled  teams  of  individuals  which  could 
better  assure  smooth  transition  from  ideas  to 
fully  supported  weapon  systems.  One  major 
insight  resulted:  it  was  shown  that  while  some 
industrial  organizations  had  been  able 
successfully  to  integrate  required  skills  into 
teams  and  had  established  integrated, 
paperless  data  bases,  the  requirement  of 
government  oversight  had  inhibiting  effects 
on  achieving  those  goals. 

Examination  of  the  possibilities  of  integrating 
U.S.  and  Canadian  technical  activities 
revealed  that  although  considerable 
integration  could  be  accomplished  specifically 
in  material  development  research,  there  were 
difficulties  in  arriving  at  joint  activities  within 
then  current  DAS  constraints. 

The  sixth  group  of  workshops  undertook  to 
integrate  all  of  the  DAS  problems  discussed 
during  the  preceding  workshops  (678 
problems  were  discussed)  and  structure  them 
to  better  enable  proposed  actions  aimed  at 
eliminating  or  alleviating  them.  In  a  sense 
these  workshops  were  tasked  to  provide  the 
road  map  for  defining  potential  change.  The 
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resulting  work  appears  in  Table  2  where  20 
broad  problem  areas  are  defined. 


Table  2 

PROBLEM  CATEGORIES  WITHIN  THE  ACQUISITION  PROCESS 


PROBLEM 

PROBLEM  CONTEXT  AND  DEFINITION 

PROGRAM  MANAGER  AUTHORITY 

D 

The  authority  of  the  program  manager  is  severely  undermined  by  the  cumulative 
effect  of  a  variety  of  intrusions  into  program  management.  In  their  totality  these 
intrusions  introduce  major  confusion  into  ongoing  activity,  and  damage  incentives 
to  do  careful  planning  for  effective  program  management,  increase  costs,  and 
delay  schedules.  Program  effectiveness  and  efficiency  are  both  casualties. 

CONTRACT  REQUIREMENTS 
DEVELOPMENT  ¥ 

Contractual  requirements  are  intended  to  specify  the  working  relationships 
between  government  and  contractor  and  to  describe  the  end  product.  At  present 
too  many  constraints  impact  ability  to  deal  properly  with  data  and  standards.  One 
consequence  is  the  inability  of  the  program  manager  to  reward  good  performance 
and  penalize  poor  performance. 

DAB-DRB  PROCESS  B 

DAB/DRB  processes  do  not  reflect  an  encompassing  treatment  across  programs, 
with  thoughtful  priority  setting.  Moreover  the  processes  do  not  adequately 
promote  the  gathering  of  relevant  information  from  the  acquisition  community 
that  should  pertain  to  DAB  /DRB  decisions.  DAB  and  DRB  are  often  in  conflict, 
not  taking  time  to  resolve  differences.  Failure  to  correct  this  situation  leads  to 
major  and  undesired  budget  surprises  for  those  who  inherit  the  impact  of  poorly 
thought  out  past  decisions. 

TECHNICAL  REQUIREMENTS 
MANAGEMENT D 

An  overly  myopic  interpretation  of  technical  requirements  mistakenly  assumes  ' 

that  they  can  be  rigorously  specified  at  program  inception  and  need  never  be 
changed.  The  unduly  restrictive  interpretation  placed  on  "requirements"  and  the 
ensuing  difficulty  in  modifying  them  as  new  knowledge  becomes  relevant,  over¬ 
constrains  program  management  and  is  a  significant  source  of  excess  time  and 
costs.  Effective  program  management  demands  that  program  managers  have  an 
approved  and  well-paved  pathway  to  change  requirements  as  conditions  warrant. 

FUNDING  INSTABILITY  B 

The  effect  of  current  program  funding  practices  is  capricious  and  destructive  of 
responsible  management  practices.  The  acquisition  budgeting  system  is  not 
conducive  to  effective  program  management,  leading  to  enormous  waste  and 
dissatisfaction. 

STATUTORY/REGULATORY 
INFLUENCES  B 

Statutory/regulatory  influences  are  often  unnecessarily  burdensome  and  do  not 
reflect  what  has  been  learned  from  decades  of  DoD  program  management. 
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Table  2  (Continued) 

PROBLEM  CATEGORIES  WITHIN  THE  ACQUISITION  PROCESS 


PROBLEM 

PROBLEM  CONTEXT  AND  DEFINITION 

COST  &  SCHEDULE  ESTIMATES  F 

Cost  and  schedule  estimates  which  have  high  leverage  in  terms  of  program 
effectiveness  and  manageability  are  seldom  realistic.  The  current  management 
style  inevitably  forces  bad  information  into  the  system  where  it  infiltrates  like  a 
virus  and  is  a  prime  source  of  program  sickness. 

INTERNATIONAL  FACTORS  C 

Current  regulations  and  plans  involving  international  military  transactions  furnish 
inadequate  guidance  for  effective  management  of  international  transactions. 

LONG  RANGE  PLANNING  B 

An  overhaul  of  the  long-range  planning  system  is  needed  to  improve  investment 
decisions  and  to  reflect  integrated  program  tradeoffs. 

USER  SUPPORT  B 

Failure  of  users  to  sustain  strong,  consistent,  long-term  support  for  programs  in 
situations  where  need  for  program  products  has  been  clearly  established  leads  to 
major  disruptions,  confusion,  and  waste.  Program  initiation  does  not  reflect 
consideration  of  system  priorities  in  relation  to  all  other  existing  or  prospective 
programs.  Long-term  plans  are  not  connected  to  short-term  decision  making. 

INADEQUACY  OF  PROGRAM  TEAM 

D 

An  effective  acquisition  program  team  supports  matching  of  demand  (which 
involves  a  whole  portfolio  of  task  imperatives  that  grow  unpredictably)  with  a 
portfolio  of  personnel  who  can  respond  capably.  Steady  increase  in  demand  in  an 
eroding  resource  environment,  accompanied  by  high  personnel  rotation; 
inadequate  personnel  experience  and  training;  and  difficulties  in  organizing  and 
sustaining  the  program  team  erodes  program  quality. 

EXECUTIVE  DECISION  AND  POLICY 
MAKERS B 

High  personnel  turnover  among  executive  decision  makers  and  policy  makers 
causes  policy  to  change  frequently.  Also  the  turnover  is  responsible  for  (a) 
continuing  misunderstanding  of  the  acquisition  system  due  to  lack  of  program 
management  training,  experience,  and  skill  at  top  DoD  and  services  management 
level;  (b)  discontinuities  and  low  quality  of  oversight,  and  (c)  inability  to  provide 
consistent  rationale  to  program  personnel  leading  to  de-motivation. 

RISK  MANAGEMENT  D 

Inadequate  contingency  planning  to  manage  cost,  performance,  and  schedule. 

There  is  no  uniform  methodology  either  to  define  or  to  handle  risk. 

INDUSTRIAL  BASE  E 

The  nation  is  losing  its  defense  industrial  base  because  it  lacks  a  consistent, 
rational,  realistic  approach  to  industry.  In  the  absence  of  articulated  policy,  de 
facto  policy  operates  in  an  undesirable  way. 

CREDIBILITY D 

No  total  quality  philosophy  operates  to  govern  program  oversight.  Inadequate 
communication  flows  through  the  system,  promoting  lack  of  trust  throughout. 

OVERSIGHT  D  &  F 

As  presently  practiced,  program  oversight  is  ill-conceived,  disorganized,  and 
replete  with  role  conflicts. 

TRANSITION  MANAGEMENT  F 

Effective  system  design  and  development  requires  phasing,  which  implies  a  need 
for  smooth  phase-to-phase  transitions.  Present  practice  involves  inadequate  phase 
definition,  insufficient  emphasis  on  transition  management,  and  poorly  managed 
transitions  from  one  phase  to  another. 
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Table  2  (Concluded) 

PROBLEM  CATEGORIES  WITHIN  THE  ACQUISmON  PROCESS 


PROBLEM 


TEST  AND  EVALUATION  A 


IMMUTABLE  B 


PROGRAM  EXECUTION  F 


PROBLEM  CONTEXT  AND  DEFINITION 


Current  government  philosophy  and  implementation  of  test  and  evaluation 
inadequately  reflects  good  management  practice.  Test  and  evaluation  practice  is 
inconsistent  with  overall  program  objectives. 


Detached,  capricious,  and  uncontrollable  high-level  acquisition  decision  processes 
are  responsible  for  micro-interference  in  programs.  Such  intrusions  translate  into 
deterioration  of  program  quality,  morale,  and  performance. 

Current  constraints  on  program  execution  favor  rigidity  when  and  where 
flexibility  is  needed;  and  they  promote  flexibility  when  and  where  rigidity  is 
beneficial;  thereby  frustrating  good  management  practices.  Structural 
impediments,  noted  for  their  frequency  and  variety,  make  program  execution 
inefficient,  and  make  it  very  difficult  for  the  program  manager  to  do  his  job. 


Table  2  was  analyzed  to  see  if  broader,  more  a  problem  area  within  one  of  these  six  groups 

generic  problem  categories  could  be  inferred.  is  indicated  by  the  letter  next  to  the  problem 

The  group  developed  the  six  generic  problem  area  title  in  Table  2. 
sets  shown  in  Table  3  below.  Membership  of 

Table  3 

GENERIC  PROBLEMS  AND  PROBLEM  CATEGORIES 


GENERIC  PROBLEMS 


A.  Test  and  Evaluation 

(1  Problem  area) 


B.  High  Level  Planning 
and  Management 

(7  Problem  areas) 


C.  International 
Cooperation 

(1  Problem  area) 


D.  Program  Office 
Effectiveness 

(6  Problem  areas) 


E.  Industrial  Base 

(1  problem  area) 


F.  Program  Execution 
Effectiveness 

(5  Problem  areas) 


PROBLEM  CATEGORIES  INCLUDED 


DAB-DRB  Process;  Funding  Instability;  Statutory /Regulatory 
Influences;  Long  Range  Planning;  Executive  Decision  And  Policy 
Makers;  Immutable 


Program  Manager  Authority;  Technical  Requirements  Management; 
Inadequacy  of  Program  Team;  Risk  Management;  Credibility; 
Oversight 


Contract  Requirements  Development;  Cost  and  Schedule  Estimates; 
Oversight;  Transition  Management;  Program  Execution 


The  third  step  was  to  examine  the  6  broad 
problem  areas  and  determine  whether  any 
of  them  negatively  influenced  any  others. 


Figure  1  shows  the  structure  of  the  problem 
categories  and  the  relationships  between 
them. 


Figure  1 


RELATIONSHIP  BETWEEN  ACQUISITION  PROBLEM  GROUPS  AND 

SUB-GROUPS 

In  Figure  1,  problems  aggravate  all  aggregation  of  workshop  wisdom 

problems  which  lie  to  their  right.  Test  and  concerning  problem  aggravation. 

Evaluation  problems  aggravate  all  other 

problems;  High  Level  Planning  and  A  FUNCTIONALLY  DERIVED 

Management  Problems  aggravate  ACQUISITION  PROCESS 

International  Cooperation  and  the  Industrial 

Base;  International  Cooperation  aggravates  Suppose  one  started  with  a  blank  sheet  of 

Program  Office  Effectiveness;  and  paper.  Could  a  very  efficient  acquisition 

Industrial  Base  Problems  aggravate  process  be  defined  which  would  permit  the 

Effectiveness  of  Program  Execution.  performance  of  functions  necessary  to  take 

ideas  and  turn  them  into  supported 
In  the  sense  that  these  major  problem  systems?  Suppose  that  acquisition  process 

groupings  (each  containing  one  or  more  consisted  only  of  activities  necessary  to  the 

broad  general  problem  area,  and  including  main  objective,  what  would  it  look  like? 

numbers  of  individual  problem  statements) 

were  found  to  be  the  most  fundamental  To  answer  those  questions,  we  convened  a 

high  level  problem  areas,  they  represent  the  series  of  five  workshops.  The  question 

asked  was:  "What  functions  must  be 
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performed  to  turn  ideas  into  fully 
supported,  fielded  weapon  systems?" 
Workshop  participants  were  drawn  from 
Industry,  from  the  major  Service 
Commands,  and  from  OSD  staff. 

At  the  time,  the  Deputy  Secretary  of 
Defense  had  made  a  change  from  prior 
policy  for  new  weapon  system 
development.  The  new  policy  would  stage 
development  -  a  prototype  system  would  be 
developed,  and  after  test  of  that  prototype, 
it  would  be  decided  if  production  quantities 
would  be  ordered.  If  it  was  decided  not  to 
produce  the  item,  the  prototype  would 
continue  to  be  used  and  data  gathered  about 
its  performance  and  capabilities.  If  at  some 
later  date,  production  quantities  were 
required,  they  would  be  procured  using  the 
material  already  in  the  archives  that  enabled 
production  of  the  prototype.  The 
workshops  took  the  new  policy  as  given, 
and  defined  a  functional  acquisition  process 
in  two  parts:  (1)  Functions  required  to 
move  from  ideas  to  prototypes;  and  (2) 
Functions  required  to  move  from 
prototypes  to  fully  supported,  fielded 
systems. 

Participants  had  a  clear  set  of  goals:  they 
wanted  to  arrive  at  a  process  which  would: 
o  Replace  Micro-Management  with 
Macro-Management  at  all  levels; 
o  Replace  the  current  "unstable, 
inflexible,  fault  intolerant"  system 
to  a  stable  and  flexible  system; 
o  Move  from  a  system  which  impedes 
telling  the  truth  to  one  which 
promotes  ethical  management; 
o  Recognize  the  fractionation  and  sub- 
optimization  implicit  in  the  present 
process  and  devise  a  system  which 
permits  synergy  between  system 
elements  and  links  systems  to 
strategy;  and 


o  Recognize  that  change  in  Defense 
budget  share  from  6%  of  GNP  to 
3%  of  GNP  will  make  it  necessary 
to  achieve  high  technology  at  much 
reduced  cost. 

Participants  were  asked  to: 

1.  Define  the  functions  involved, 

2.  Group  appropriate  functions 
together, 

3.  Develop  relationships  between 
functional  groups, 

4.  Define  attributes  of  organizations 
focused  on  efficient  performance  of 
the  system  functions, 

5.  Devise  relationships  between 
organizational  elements  within  the 
acquisition  process  and  other 
Governmental  and  industrial 
organizations  (and  organizational 
elements)  which  interact  with  them. 

MOVING  FROM  IDEAS  TO 
PROTOTYPES 

The  acquisition  process  functions  they 
believed  necessary  for  this  portion  of  the 
complete  system  development  process 
contained  108  individual  functions  grouped 
within  7  general  functional  categories.  The 
functional  categories  and  the  sequence  of 
performance  of  functions  are  shown  in 
Figure  2. 
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Figure  2 


Figure  2  shows  that  Need  Identification 
Functions  must  be  completed  prior  to 
Analyzing  Needs.  An  Acquisition  Strategy 
can  be  developed  only  after  needs  have 
been  developed  and  analyzed.  Then,  the 
strategy  can  be  implemented  concomitantly 
with  design,  test  and  evaluations  functions. 
When  those  have  been  completed,  the 
output  can  be  achieved.  Figure  2  indicates 
that  management  and  organization  functions 
are  performed  continuously  throughout  the 
process  of  taking  ideas  to  prototype 
equipment. 

Participants  believed  that  Identification  of 
Need  and  Analysis  of  Need  required 
different  kinds  of  skills  than  did  the 
remainder  of  the  functional  process  (see 
below). 


developed  a  total  of  312  statements  they 
felt  were  "functions  to  be  performed"  in  the 
process  of  moving  from  prototype  weapon 
systems  to  fully  supported,  deployed 
weapon  systems.  Participants  were  asked  to 
group  all  functions  into  a  set  of  over¬ 
arching  functional  groups.  Four  major, 
temporally  related,  functional  action 
groupings  resulted  from  this  activity. 
Figure  3  indicates  the  groups  and  the  way 
they  relate  to  each  other.  Figure  3 
indicates  that  pre-production  activities  must 
be  completed  prior  to  entering  into  the 
production  phase,  and  that  deployment  and 
sustaining  functions  follow  after  production 
functions  have  been  accomplished. 

SKILLS  REQUIRED  TO  PERFORM  THE 
FUNCTIONS 


MOVING  FROM  PROTOTYPES  TO 
SUPPORTED  OPERATIONAL  SYSTEMS 

We  used  the  same  process  previously  used 
to  derive  functions  required  to  move  from 
ideas  to  prototype  systems.  During  the 
course  of  the  workshop,  participants 


Participants  analyzed  the  kinds  of  personnel 
skills  required  to  perform  the  functions  they 
defined,  deciding  on  7  skill  sets  which 
would  cover  all  functions  involved.  The 
skill  sets  and  a  brief  definition  of  what 
those  skill  sets  entail  is  shown  in  Table  4 
on  the  next  page. 
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Figure  3 

FUNCTIONS  NECESSARY  TO  PROGRESS  FROM  PROTOTYPES 
TO  FULLY  SUPPORTED  SYSTEMS 

Participants  used  the  skill  set  definitions  to  insights  into  how  best  to  construct  a 

determine  skill  set  mixes  required  to  functionally  responsive  organizational 

perform  the  functions  within  the  defined  structure  for  moving  from  prototypes  to 

acquisition  process.  That  analysis  provided  fully  supported  systems. 

Table  4 


SKILLS  REQUIRED  FOR  THE  FUNCTIONAL  ACQUISITION 

PROCESS 


SKILL  SET 

NARRATIVE  DESCRIPTION 

Management  Skill 

Set 

Primarily  skilled  in  managing  diverse  disciplines;  capable  of 
understanding  details  of  significant  factors  in  those  disciplines 
integral  to  the  program. 

Engineering  Skill 

Set 

Highly  expert  in  engineering  disciplines;  understands  principles 
of  other  disciplines  integral  to  development,  production,  and 
sustaining  activities. 

Logistic  Skill  Set 

Highly  skilled  and  having  in-depth  knowledge  of  the  principles 
and  practices  of  sustainment  activity  at  both  tactical  locations, 
and  in  production  and  program  management  sites. 

Procurement  Skill 

Set 

Procurement  professional  with  detailed  understanding  of 
contracting  principles  and  practices;  capable  of  using  technical 
inputs  from  other  program  skill  sets  to  achieve  procurement 
objectives. 

Finance  Skill  Set 

Deep  understanding  and  knowledge  of  financial  activities  and 
capacity  to  use  technical  inputs  from  other  program  skills  to 
achieve  timely  financial  support  for  program  activities. 

Manufacturing  Skill 
Set 

Expert  in  technology  and  practice  of  manufacture  both  of 
singular  items  and  production  quantities  of  complex  technical 
sub-systems  and  systems  Capable  of  defining  requirements  for 
production  facilities  and  individual  production  components  for 
both  "in-use"  and  "new"  production  techniques 

User  Skill  Set 

Seasoned,  skilled  in  theory  and  practice  of  military  tactical 
operations.  Capable  of  translating  tactical  understanding  into 
useful  input  for  program  office  team  specialists  to  help  define 
policies  and  actions  which  keep  the  program  focused  on  its 
military  objectives. 

The  skills  analysis  revealed  the  need  for 
application  of  multiple  skills  to  perform 
almost  all  of  the  functions  necessary  to  move 
from  ideas  to  fielded,  supported  systems.  In 
fact,  the  analysis  indicated  specific  kinds  of 
multi-functional  groups  discussed  in  the 
concurrent  engineering  workshops,  system 
engineering  round  tables,  and  other 
workshops  in  the  fifth  workshop  group 
enumerated  in  Table  1  above.  There  was 
however,  a  considerable  difference  in  the 
numbers  of  each  kind  of  skill  set  required  at 
different  times  during  the  process  of  moving 
from  ideas  to  deployed  systems. 


ORGANIZING  TO  REALIZE 
FUNCTIONAL  ACQUISITION 

Participants  became  convinced  that  there  were 
three  major  process  components: 

1.  Defining  requirements  and 
maintaining  their  currency; 

2.  Responding  to  the  requirements  by 
producing,  testing,  and  deriving  an 
envelope  of  operational  integrity  for  a 
prototypical  system;  and 


3.  Producing,  deploying  and  sustaining 
operational  systems  derived  from  the 
well  understood  prototype. 

Participants  believed  each  process  component 
needed  its  own  kind  of  organizational 
philosophy. 

o  The  first  set  of  functions  is 
dominated  by  user  skills.  The  analysis 
indicates  that  they  should  be  performed  by 
highly  skilled  technical  and  military 
operational  staff  members  who  would  review 
technology  and  operational  needs,  and  when 
action  was  indicated,  would  recommend  that 
a  new  weapon  capability  be  developed. 
Participants  believed  that  such  work  would  be 
best  performed  under  the  auspices  of  the  Joint 
Chiefs  of  Staff.  Performance  of  these 
functions  would  result  in  a  defined 
requirement  emerging  from  an  on-going 
analytical  process  within  JCS.  The  JCS 
analysis  would  be  formally  stated,  once 
coordinated  with  appropriate  operational 
commands.  The  service  development 
activities  would  respond  to  the  requirement 
with  formal  development  proposals. 
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With  the  proposals  received,  JCS  and  the 
service  materiel  developer  would  recommend 
issuance  of  a  directive  to  develop  the  new 
weapon  capability  as  set  forth  in  the 
development  proposal.  The  directive  would 
be  issued  by  an  oversight  group  who  would 
review  both  the  need  and  the  response  to 
insure  that  the  planned  development  would 
meet  national  priorities  and  would  be  of 
reasonable  cost.  The  issuance  of  the  formal 
directive  would  establish  a  program  office 
and  authorize  that  office  to  produce  the 
prototype  system  in  response  to  the 
requirements  defined. 

o  The  second  set  of  functions  is 
dominated  by  engineering  and 
manufacturing  skills.  Participants  felt 
industrial  organizations  provided  models  for 
performing  these  functions.  Examples  were 
the  Lockheed  "Skunk  Works"  which  had 
produced  the  RB-70  and  the  U-2, 
McDonnell-Douglas's  "Phantom  Works" 
which  had  pioneered  use  of  soft  tooling  for 
aircraft  structural  builds,  and  Burt  Rutan's 
"Scaled  Composites"  organization  which  had 
produced  the  Voyager  and  other  experimental 
aircraft.  Participants  believed  that  an 
autonomous,  fully  responsible  program 
manager  and  a  program  management  staff 
could  efficiently  and  effectively  perform  the 
Government's  direction  and  Industrial 
oversight  functions.  Numerous  examples 
were  cited  as  organizational  and  staff 
functional  models  for  performing  the 
functions  within  this  portion  of  the  redesigned 
process. 

At  the  point  when  a  prototype  system  has 
been  developed,  and  sufficiently  tested  to 
have  provided  an  envelope  of  operational 
capability,  a  second  directive  would  be  issued 
containing  one  of  two  options:  (1)  Continue 
to  use  the  prototype  system  and  refine  it  as 
indicated  through  that  use;  or  (2)  Proceed  to 


produce,  deploy  and  sustain  some  numbers  of 
the  existing  system.  The  factors  which  would 
drive  the  choice  were:  evaluation  of  the 
prototype  system  against  up-dated 
requirements,  and  the  estimated  cost  and 
schedule  involved  in  the  production  process. 
Given  either  choice,  the  Program  Office 
would  take  appropriate  action  to  reform  itself 
as  required.  The  appropriate  oversight 
authority  would  be  convened  to  issue  the 
directive. 

o  The  third  set  of  functions  is 
dominated  by  management,  engineering  and 
manufacturing  skills.  Participants  once  again 
stated  that  there  were  models  within  both 
industry  and  Government  to  serve  as  guides 
in  organizing  and  contracting  for  performance 
of  the  necessary  functions  involved  in 
proceeding  from  operating  prototypical 
systems  to  fully  supported  deployed  weapons. 

When  production  items  have  been  deployed 
and  sustained  under  operational  conditions  for 
a  time  sufficiently  long  to  judge  their 
effectiveness  and  operating  economies,  an 
additional  oversight  point  arises.  The 
Program  Office  would  again  request 
convening  of  the  oversight  authority  to 
evaluate  the  Field  Commander's  assessments 
of  the  system,  and  the  assessment  of  the 
permanent  requirements  organization  within 
the  JCS.  The  authority  would  determine 
whether  a  new  or  revised  system  was 
necessary.  The  JCS  organization  would  be 
required  to  certify  that  a  new  need  exists 
which  cannot  be  met  by  the  existing  systems 
and  the  field  commanders  would  be  required 
to  confirm  that  presendy  operational  systems 
could  not  meet  those  need.  Again,  industrial 
organizations  provide  models  for  performing 
these  oversight  functions  through  their  Boards 
of  Directors  who  authorize  program  changes. 
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In  short,  participants  designed  an  organization 
which  would  be  autonomous,  with  oversight 
exercised  sparingly,  and  only  when  events 
made  that  oversight  essential  to  maintain  the 
integrity  of  the  acquisition  process.  Two 
directives  issued  by  the  oversight  agency 


would  assign  responsibility  and  authority  and 
would  provide  assurance  of  necessary 
resources  subject  only  to  the  limitations  of  the 
funding  authorization  by  Congress.  The 
redesigned  acquisition  process  appears  in 
Figure  4  below. 


INDEPENDENT  PERMANENT 
ORGANIZATION  WORKING  WITH 
THE  JOINT  CHIEFS  OF  STAFF 


PROGRAM  OFFICE 
FOR  PROTOTYPE 
DEVELOPMENT  ANO 
TESTING 


SECOND  DECISION  POINT 


AUTHORIZE  FUNDING  FOR 
PRODUCTION.  OR  CONTINUE 
PROTOTYPE  OPERATION 


PROGRAM  OFFICE  FOR 
PRODUCTION,  DEPLOYMENT 
AND  SUPPORT 


FUNCTIONAL  FLOW 


Figure  4 

THE  REDESIGNED  ACQUISITION  PROCESS 


PROBLEMS  WITH 

INSTITUTIONALIZING  FUNCTIONAL 
ACQUISITION 

After  the  functional  acquisition  process  had 
been  re-designed,  a  workshop  was  convened 
to  evaluate  its  effect  in  reducing  or 
eliminating  the  678  problems  (which  had 
been  defined  as  present  within  the  acquisition 
process  prior  to  Congress  enacting  the 
Federal  Acquisition  Streamlining  Act  of  1994 
in  October  of  last  year).  The  participants  in 
that  workshop  looked  carefully  into  changes 


in  acquisition  policy  resulting  from  Defense 
Department  actions,  and  from  the  various 
Defense  Management  Reviews  (DMR's) 
issued  by  DoD. 

It  was  their  conclusion  that  internal  DoD 
actions  had  eliminated  a  substantial  number  of 
the  problems  already,  and  that 
institutionalizing  the  functional  acquisition 
process  would  ameliorate  or  eliminate  a  great 
number  of  the  problems  which  remained.  In 
their  judgement,  612  of  the  678  problems 
identified  would  be  or  had  been  eliminated. 
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No  problems  remained  in  5  of  the  problem 
areas  defined  in  Table  3:  DAB-DRB  Process, 
Transition  Management,  Test  and  Evaluation, 
Credibility,  and  Oversight. 

But  what  about  the  remaining  66  problems ? 

Table  5  indicates  the  problems  remaining 
unaffected  by  institutionalizing  a  functional 
acquisition  process.  Most  of  the  problems 
which  remain  have  to  do  with  the  process  of 
oversight  and  the  way  in  which  it  must  be 
exercised  by  law. 

Over  the  past  30  years,  our  society  has 
developed  an  aversion  to  error  that  has  lead  to 
an  unwillingness  to  accept  mistakes  no  matter 
the  reasons  for  making  them.  The  record  of 
predicting  the  future  is  dismal.  The  Chairman 


of  the  Federal  Reserve  Board  has  publicly 
stated  that  models  which  were  formerly  useful 
in  predicting  the  economic  state  of  the  nation 
have  lost  their  utility  over  the  years.  The 
inability  of  anyone  to  predict  the  future 
accurately  is  demonstrated  daily  when  some 
prediction  made  years  ago  fails  to  materialize 
as  expected.  The  problem  is  that  change  is  so 
rapid  there  is  little  assurance  that  any  long 
term  projections  will  be  accurate  in  the  end. 
The  unwillingness  to  accept  the  inevitability 
of  humans  making  mistakes  has  led  to  a 
peculiarly  structured  system  which  affects  all 
of  us.  The  logic  of  that  system  appears  to  be 
as  follows: 

1.  Mistakes  are  not  inevitable;  perfection  is 
achievable  through  proper  use  of  technology 
and  paying  attention  to  lessons  learned. 


Table  5 

PROBLEMS  LEFT  UNSOLVED  BY  A  FUNCTIONALLY  DESIGN 

ACQUISITION  PROCESS 


GENERIC  PROBLEM  AREA 


PROGRAM  MANAGER 
AUTHORITY 


CONTRACT  REQUIREMENTS 
DEVELOPMENT 


SPECIFIC  PROBLEMS  REMAINING 


O  Political  motives  in  the  decision  process 
O  No  one  can  say  "Go"  but  everyone  can  say  "Stop" 

O  Interference  from  congressional  oversight 
O  Proliferation  and  lack  of  accountability  of  ankle  biters 
O  Political  influences  beyond  the  program  managers’  control 
O  Too  many  nay-sayers  ...  In  the  review  chain 
O  Tendency  not  to  surface  problems 

O  Too  many  participants  can  stop  or  slow  process  without  responsibility  for 
delivering  the  product 

O  Adverse  impact  of  well  meaning  but  ineffective  attempts  improve  the 
process 

O  Data  requirements 

O  Failure  to  adequately  describe  performance  verification  and  validation 
process  by  which  success  is  measured 


TECHNICAL  REQUIREMENTS 
MANAGEMENT 


FUNDING  INSTABILITY 


o 

o 


o 

o 


Inability  to  synthesize  a  design  the  first  time 
Changes  in  policy  and  specifications 


Year-to-year  instabilities  to  budget  and  procurement  quantities 
Lack  of  fiscal  planning 


STATUTORY-REGULATORY 


O  Constraining  procurement  laws  and  acquisition  regulations 
O  Excessive  procurement  laws  and  regulations 

O  Mandate  for  competition  of  small  business  that  may  be  unqualified  to 
participate 
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INFLUENCES 

O  Lack  of  regulation  and  historical  approach  cleansing 

O  Illogical  competition 

O  Changes  in  policy  and  specifications 

LONG  RANGE  PLANNING 

O  Acquisition  process  considers  program/requirements  on  only  an 
individual  basis  without  considering  larger  investment  context  and 
trade-off 

O  Lack  of  or  undisciplined  strategic  planning 

O  Lack  of  clear  military  strategy  and  quantitative  military  requirements 

O  Short-term  planning  dominates  decision  making  process 

USER  SUPPORT 

O  Lack  of  priority  by  acquisition  organization  for  weapon  systems 

O  Lack  of  strong,  consistent,  and  long-term  user  support  for  smart 
munitions  programs 

INTERNATIONAL  FACTORS 

O  Inadequate  foreign  sales  planning 

O  U.S.  Security  and  custom  regulation  not  consistent 
with  international  co-development 

Table  5  (Continued) 

PROBLEMS  LEFT  UNSOLVED  BY  A  FUNCTIONALLY  DESIGNED 

ACQUISITION  PROCESS 


GENERIC  PROBLEM  AREA 

SPECIFIC  PROBLEMS  REMAINING 

COSTAND  SCHEDULE 
ESTIMATES 

O  Unrealistic  program  plans/schedules  and  associated 
funding  profiles 

O  Pressure  for  unrealistic  schedule,  cost,  and 
performance 

O  Ineffective  cost  estimating  up-front 

O  Competitive  pressures  lead  to  unrealistic 
expectations 

O  Assessment  of  program  cost  risk  by  DoD  is 
inadequate 

O  Failure  of  contractors  to  propose  realistic  costing  to 

RFP's  (buying  in) 

O  The  government  forces  contractor  to  buy  in  thereby  increasing  the  risk 

O  Lack  of  government  understanding  of  the  cost  of 
procuring  many  smart  munitions  programs 

INADEQUACY  OF  PROGRAM 
TEAM 

O  Loss  of  program  focus  due  to  program  personnel  rotation 

O  Lack  of  contractor's  ability  to  provide  people  resources  as  required 

O  Lack  of  acquisition  training  of  experience  of  superiors 

O  Inadequate  resources  outside  the  program  office  (doing  more  with 
less) 

O  Barriers  erected  between  defense  and  non-defense  divisions  of 
companies  and  sectors 

IMMUTABLE 

O  Instability  of  dod  and  congressional  support  for  programs 

O  Abrogation  of  commitment  at  all  levels 

O  Its  never  over 

O  there  is  no  agreement  between  the  executive  branch  and  the  congress  on 
the  long-term  budget  projection 

O  Congressional  authorization  and  appropriation  process 

O  Congressional  mistrust  and  meddling  and  language 

PROGRAM  EXECUTION 

O  No  OSD/Service  policy  on  concurrent  engineering 

Q  Inability  to  award  timely  contracts  due  to  external  controls 

EXECUTIVE  AND  POLICY 
MAKERS 

O  Leadership  high  turnover  rate 
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RISK  MANAGEMENT 


O  Failure  to  know  how  to  respond  to  risks  even  when  known  (risk/pen¬ 
alty/profit) 

O  Imprecise  risk  management  methodologies 

O  Management  of  the  dod  acquisition  process  is  not  disciplined  enough 
O  Lack  of  early  management  focus 


Table  5  (Concluded) 

PROBLEMS  LEFT  UNSOLVED  BY  A  FUNCTIONALLY  DESIGN 

ACQUISITION  PROCESS 


GENERIC  PROBLEM  AREA  SPECIFIC  PROBLEMS  REMAINING 


O  Loss  of  industrial  base  (inadequate  r&d) 

O  OSD  is  a  small  customer  of  the  general  industrial  base 
O  Outward  migration  of  investment  capital  and  skilled  people 
O  DoD  is  unwilling  to  fund  industrial  base  improvement  program 
O  Intervention  is  not  an  administration  policy 

O  Their  present  acquisition  policies  contain  conflicting  goals  making  it 
INDUSTRIAL  BASE  virtually  impossible  to  strengthen  the  base 

O  Lack  of  a  clear  understanding  of  the  consequences  of  some  of  the 
perceived  industrial  base  problems 

O  Lack  of  agreement  on  the  crucial  or  core  elements  of  the  industrial  base 
that  must  be  sustained 

O  Failure  to  consider  the  industrial  base  early  in  the  acquisition  process 
O  DoD  Fiscal  management  structure  does  not  support  and  strengthen  the 
industrial  base  (unit  cost  policy) 

O  Need  to  formalize  or  institutionalize  consideration  of  industrial  base 


2.  Any  mistakes  which  are  made  probably 
result  from  individuals  attempts  to  gain  their 
own  ends. 

3.  Society  must  guard  against  making 
mistakes  first  by  structuring  the  environment 
to  minimize  the  opportunity  to  take  incorrect 
action  and  secondly  to  provide  continuing 
oversight  which  will  insure  that  actions  taken 
are  in  accordance  with  the  structure. 

4.  If  mistakes  are  seen  to  have  been  made, 
those  who  made  them  will  be  held 
accountable  regardless  of  whether  or  not  the 
action  was  eminently  reasonable  at  the  time  it 
was  taken. 

The  major  difficulty  with  changing  the  way 
acquisition  is  structured  results  from  the  need 
to  prevent  any  kind  of  error  or  misfeasance  in 
expending  tax  monies.  In  the  name  of  fiscal 


integrity,  oversight  and  reporting 
requirements  are  levied  to  provide  that 
assurance.  To  reinforce  that  point  of  view,  a 
careful  review  of  the  testimony  which  led  to 
the  creation  and  passage  of  the  latest 
Acquisition  legislation  reveals  those  same 
concerns  shaped  the  bill.  The  record  of 
testimony  reflects  the  fears  which  have 
created  and  continue  to  mitigate  against 
solving  the  problems  listed  in  Table  5. 


WHAT  CAN  BE  DONE  TO  IMPROVE 
ACQUISITION  PERFORMANCE? 

The  major  difficulty  in  preventing  technical 
and  financial  mistakes  results  from  the 
requirement  to  forecast  outcomes  accurately 
very  far  in  advance  of  the  actual  events.  The 
closer  a  technical  event  is  to  taking  place,  the 
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more  likely  it  is  that  predictions  made  about 
that  event  will  be  in  error  simply  because  so 
many  things  which  affect  the  outcome  of  a 
process  can  change  rapidly. 

If  we  agreed  that  (a)  planning  five  years  or 
more  into  the  future  can  best  be  done  in 
general  terms  and  that  (b)  detailed,  specific 
forecasts  of  events  which  are  scheduled  to 
occur  in  the  long  term  are  likely  to  be  in  error 
simply  because  of  what  insurance  companies 
call  "acts  of  God",  then  a  great  many 
difficulties  rooted  in  the  inability  to  forecast 
perfectly  would  be  eliminated. 

Most  technologists  and  engineers  can  see  the 
shape  of  things  to  come  quite  well  -  and  some 
can  even  project  within  a  few  years  when 
technologies  can  (will)  become  useful  in  ways 
that  can  help  create  better  materiel  for  our 
armed  forces.  Few,  however,  can  detail  those 
projections  to  permit  accurate  (errorless) 
planning  even  as  much  as  one  year  into  the 
future. 

The  workshops  seem  to  tell  us  that  we  could 
improve  acquisition  performance  greatly  by 
providing  oversight  sparingly  but  at  crucial 
times;  and  by  recognizing  that  there  are  short 
time  limits  to  long-term  planning.  Taking 
those  actions  can  help  us  all.  Coiporate 
America  is  having  to  follow  the  new  thinking 
about  oversight  and  planning;  perhaps  there  is 
a  lesson  in  that  for  all  of  us. 
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Can  “IPD”  Work  in  DoD? 

(How  About  Lean  Practices?) 

Lieutenant  Colonel  Mikael  S.  Beno,  USAF 

ABSTRACT 


In  order  for  the  Department  of 
Defense  (DoD)  to  accomplish  its 
mission,  new  ways  of  doing  business 
must  be  created.  Manpower 
reductions  alone  dictate  this. 
Integrated  Product  Development 
(IPD)  may  be  one  way  to  require 
managers  to  face  this  fundamental 
paradigm  shift.  This  paper 
discusses  IPD  tenets  (empowerment, 
product  focus,  up-front  planning, 


right  person/place/time,  seamless 
management  tools,  and  teamwork 
and  communications)  and  their 
relationship  to  lean  engineering 
principles.  In  addition,  this  paper 
answers  the  questions,  “How  do  IPD 
tenets  and  lean  practices  result  in 
better  management  and  improved 
products,  and  what  are  the  risks 
and  manpower  implications?” 
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CAN  IPD  WORK  IN  DOD  (HOW 
ABOUT  LEAN  PRACTICES)? 

INTRODUCTION 

The  purpose  of  this  paper  is  to  analyze 
and  discuss  Integrated  Product  Development 
(IPD)  tenets  and  their  relationship  to  lean 
engineering  principles  as  discussed  in  "The 
Machine  That  Changed  the  World."  How  do 
IPD  tenets  and  lean  practices  result  in  better 
management  and  improved  products,  and 
what  are  the  risks  and  manpower 
implications?  Finally,  as  you  read  through 
this  article,  answer  the  questions  "What  is  a 
lean  organization"  and  "What  are  the 
limitations  of  employing  lean  practices  in 
DoD?" 

HISTORY 

Air  Force  Materiel  Command  (AFMC) 
has  sent  out  the  word  that  all  centers  will  use 
the  concepts  of  IPD,  but  there  appears  to  be 
a  wide  gulf  on  progress  toward 
implementing  it  as  well  as  explaining  its 
tangible  values.  The  reactions  to  this  "new" 
concept  have  been  varied,  but  more  often 
than  not  the  most  common  are  some 
combination  of  "IPD  is  just  another 
buzzword,  it's  just  plain  old  common  sense, 
or  IPD  is  really  nothing  new." 


Lean  engineering  and  manufacturing 
principles  are  the  precursors  of  IPD  and  are 
a  major  foundation  on  which  the  tenets  of 
IPD  were  developed.  Information  in  this 
article  on  lean  engineering  and  car 
manufacturing  concepts  was  taken  from  the 
book  "The  Machine  that  Changed  the 
World." 

Lean  car  companies  show  a  reduction  of 
engineering  design  hours  on  major  projects 
by  40%,  schedule  by  25%,  and  improved 
quality  when  comparing  data  with 
companies  using  traditional  methods.  How 
do  they  accomplish  these  significant 
improvements,  and  do  the  factors  that  allow 
this  apply  to  the  DoD  acquisition  process? 
What  is  the  impact  of  not  fully  integrating 
IPD  into  the  DoD  culture?  Is  the 
acknowledgment  of  a  paradigm  shift  a 
necessary  precursor  to  success? 

Piecemeal  imitation  of  successful 
organizations  is  a  common  reason  for 
failure  in  efforts  requiring  significant 
change  (Wilkins). 

TENETS  OF  IPD 

IPD  is  comprised  of  no  less  than  seven 
separate  tenets.  All  of  these  tenets 
(empowerment,  product  focus,  up-front 
planning,  right  person/place/time,  seamless 
management  tools,  integration,  and 
teamwork  and  communication)  merge  to 
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result  in  a  new  approach  to  doing  business, 
in  essence,  a  cultural  change  (which  AFMC 
says  itself  is  a  tenet).  Each  tenet,  what  it 
really  means,  and  its  relationship  to  lean 
engineering  will  be  discussed. 

EMPOWERMENT 

What  is  an  empowered  leader?  How  do 
you  know  if  you  are  empowered?  Why  do 
you  want  to  empower  your  people? 
Empowerment  requires  at  least  two 
confident  and  capable  people  and  is  a 
function  of  a  supervisor's  trust  in  the 
employee's  ability  as  well  as  his  ability  to 
relinquish  control.  This  is  a  most  difficult 
concept  for  managers  in  large  bureaucracies, 
particularly  when  it  is  expected  that 
managers  themselves  are  held  accountable 
for  every  facet  of  a  project.  This  has  resulted 
in  a  form  of  micro-management  and  a 
reluctance  to  allow  people  to  function 
without  constant  reviews  and  approvals. 
This  mentality  applies  to  contract 
management  as  well.  We  continually 
conduct  reviews  and  audits  of  our 
contractors.  Why?  Don't  we  trust  them? 
You  can  not  expect  to  have  any  significant 
improvements  in  productivity  without  trust. 
If  we  were  to  trust  our  contractors,  and 
minimized  reviews,  audits,  and  the  like, 
what  would  be  the  impact  on  manpower, 
particularly  within  the  contract 
administrative  areas?  Until  we  change  the 
reviewer  mentality  that  currently  exists 
among  all  of  us  and  begin  to  put  a  much 


higher  burden  on  the  contractor  for  product 
success,  it  will  be  difficult  to  change  the 
process,  or  the  level  of  productivity.  Lean 
companies  trust.  The  trust  is  based  on  clear 
expectations  expressed  in  contract  language. 
The  first  step  toward  being  lean  is  to 
empower  the  contractor  to  do  the  job  and  get 
out  of  his  hair.  A  necessary  condition  for 
trust  is  to  write  better  RFPs. 

An  empowered  person  is  not  a 
coordinator  of  functional  inputs  (which 
many  program  manager's  are  today)  but  has 
real  control  of  the  project.  Do  you  as  the 
project  leader  write  the  evaluation  reports  of 
the  functionals  on  your  team  (isn't  this  the 
essence  of  real  control)?  The  simple  answer 
for  most  is  no.  In  fact,  a  recent  AFMC  500 
series  regulation  provided  guidance  to 
centers  that  team  functionals  would  be  rated 
by  the  functional  three  letter,  not  the  IPT 
team  chief.  This  is  not  lean  and  it  is  not  IPD. 
The  primary  reason  for  this  is  that  as  organic 
manpower  resources  diminish,  the  ability  to 
put  full  time  functionals  (i.e.,  engineers, 
data/configuration  managers,  contracting 
officers,  logisticians,  etc.)  on  teams  will 
decrease.  People  will  be  spread  thinner  over 
more  contracts  and  the  ability  to  justify 
rating  personnel  by  IPT  team  leaders  will 
drop.  Nonetheless,  a  return  to  matrix 
management,  for  whatever  reason,  is  not 
IPD. 

If  you  are  empowered,  your  boss  gives 
you  decision  making  authority 
commensurate  with  your  experience  level 
(decentralization).  Look  for  examples  in  the 
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management  of  your  program.  What 
functions  are  you  not  the  final  decision 
authority  on  in  which  you  should  be?  Go  to 
your  boss  and  get  them  changed.  If  you  find 
resistance,  the  primary  cause  will  most 
probably  be  a  failure  to  understand  IPD  and 
the  resultant  paradigm  shift  away  from 
centralized  decision  making.  Managers  are 
still  afraid  of  the  ramifications  of  not  being 
able  to  answer  their  boss's  questions. 

Why  are  we  trying  to  foster  empowered 
leaders?  With  decision  making  made  at  the 
lowest  possible  level,  you  can  expect  to  see 
fewer  middle  managers  and  a  reduction  in 
preparing  and  delivering  time  consuming 
briefings.  This,  both  within  the  government 
and  with  our  contractors,  can  have  a 
significant  impact  on  manpower.  In 
addition,  senior  leadership  will  be  not  only 
developing  the  leaders  of  the  future,  but  in  a 
study  conducted  on  an  individual's  choice  of 
more  pay  or  greater  authority  on  the  job,  a 
vast  majority  took  the  additional  authority. 
People  want  to  feel  ownership  and  be 
accountable,  and  this  makes  for  a  happier 
and  more  productive  worker.  The  bottom 
line-IPD  means  empowered  employees  and 
contractors,  and  DoD  needs  to  brainstorm 
areas  in  which  to  implement  this  tenet. 

PRODUCT  FOCUS 

Ask  yourself  the  question,  what  were  the 
functionals  most  interested  in  under  the  old 
system,  namely  pure  matrix  management? 
What  was  the  focus?  More  times  than  not 


wasn't  it  that  their  CDRL,  or  their  SOW 
paragraph  got  into  the  contract?  Were  they 
interested  in  what  some  other  functional  was 
doing,  or  a  problem  not  associated  with  their 
area?  The  answer  to  all  of  these  questions  is 
probably  no  because  rewards  and  penalties 
were  (and  continue  to  be)  oriented  toward 
individual  accomplishment,  not  the  team. 
The  term  "functional  stove  piping"  was 
coined  to  indicate  this  mentality.  Under  IPD, 
the  primary  focus  is  the  product  not  the 
functional  ritual  of  submitting  their  SOW 
paragraph  or  CDRL.  This  new  focus  starts 
with  each  individual  of  the  team 
understanding  what  the  customer  really 
wants.  In  Steven  Covey's  book,  "The  7 
Habits  of  Highly  Effective  People,"  his  habit 
number  two  is  "begin  with  the  end  in  mind." 
What  are  we  really  trying  to  do-what  is  the 
bottom  line?  It  is  not  to  develop  paper  but  to 
put  reliable  hardware  in  the  hands  of  our 
defense  forces.  In  order  to  develop  and  field 
the  product  to  the  users  in  a  timely  manner, 
it  is  absolutely  critical  that  all  team  members 
understand  what  the  customer  wants,  and 
that  brings  us  to  the  "requirements"  process. 
If  this  process  is  broken,  then  the  acquisition 
phases  (Post  milestone  1)  will  be  prone  to  a 
wide  variety  of  future  problems. 

The  requirements  process  is  in  need  of 
repair.  In  one  study  chartered  to  look  at  this 
problem,  some  of  the  major  findings  were: 

1)  Little  or  no  pre  milestone  0  work  is  being 
done  (MAA,  MNA)  resulting  in  products 
being  developed  because  technology  exists, 
not  because  there  is  an  operational  need.  We 
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field  products  with  often  no  quantifiable 
understanding  of  its  impact  on  the  outcome 
of  battle?  This,  of  course,  should  be 
determined  prior  to  expending  post 
milestone  1  resources. 

2)  The  user  does  not  have  the  necessary 
acquisition/technical  skills  or  budget  to 
accomplish  pre  milestone  1  activities  alone. 
This  results  in  a  need  to  clearly  define  roles 
which  current  regulations  fail  to  do.  It  is 
clear  that  users  are  responsible  for  all  pre 
milestone  1  tasks,  but  who  should  be  doing 
them?  Confusion  results  causing  a  variety 
of  ills  (false  starts,  team  disunity,  etc.) 

3)  Centers  can  not  "satisfy  the  customer"  if 
requirements  are  not  clear.  The  AFMC 
product  and  logistics  centers  are  the 
customers  of  the  requirements  process.  The 
users  are  the  suppliers.  ORDs  must  be 
delivered  on  time,  provide  for  trade  space 
(no  solutions),  and  be  traceable  (which 
requires  common,  user  friendly  traceability 
tools)  back  to  the  functional  objectives  of 
the  MAA.  Sadly,  for  most  programs,  these 
issues  are  symptomatic.  How  do  we  change? 

Our  relationship  with  our  customer  must 
be  clarified.  Current  regulatory  guidance  is 
not  clear.  An  overarching  MOA  between 
acquisition  centers  and  their  customers 
which  clarifies  roles  is  needed,  and 
individual  team  charters  talking  to  products, 
with  the  "who's"  and  "when's"  attached,  are 
first  steps.  Team  training  with  all  the  players 
(SPO,  user(s),  and  contractors)  using  the 
same  game  plan  is  clearly  needed. 


UP-FRONT  PLANNING,  RIGHT 
PERSON/PLACE/TIME,  AND 

INTEGRATION 

How  would  you  score  DoD  on  up-front 
planning?  For  example,  how  would  you 
explain  why  there  are  so  many  ECPs, 
schedule  slips,  cost  overruns,  or 
misunderstandings  resulting  in  claims  from 
our  contractors  or  complaints  from  your 
users? 

The  primary  cause  of  schedule  delays  and 
cost  overruns  is  the  failure  to  properly  plan 
acquisitions  (or  is  it  changing  requirements, 
arguably  a  subset  of  planning).  In  order  to 
solve  this  problem,  more  up  front  planning 
is  required,  but  what  specifically?  What  is 
the  primary  product  of  a  center?  Isn't  it  the 
request  for  proposal  (RFP)?  And  what  could 
be  used,  both  as  a  way  of  ensuring  the 
contractor  is  capable  at  contract  award  and 
as  a  way  to  manage  the  program?  IPD 
suggests  seamless  management  tools  are  the 
answer!  These  tools  are  covered  in  more 
detail  later  on. 

Historically,  there  have  been  many  cases 
where  this  up-front  planning  has  simply  not 
been  done.  RFPs  are  prepared  by  junior 
officers  who  have  little  experience. 
Documents  are  reviewed  at  the  end  (murder 
boarded)  versus  building  in  quality.  This 
usually  results  in  changes  and  increases  in 
the  time  it  takes  to  get  out  an  RFP  (and 
consequently  providing  the  product).  Time 
phasing  of  activities  is  not  considered  and 
many  times  arbitrary  dates  are  set  for 
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contract  periods  of  performance. 
Documentation  is  either  not  prepared,  not 
provided  in  time,  incomplete,  or  prepared  by 
a  sub-optimal  team.  These  symptoms,  one 
might  guess,  could  be  found  throughout 
DoD. 

People  who  understand  both  the 
requirements  and  acquisition  processes,  and 
the  role  of  system  engineering  are  critical 
and  valuable  resources  in  DoD.  Few  of 
these  people  exist.  This  means  that  in  view 
of  the  fact  that  DoD  does  not  have  the 
resources  to  allocate  a  full  person  to  each 
team,  teams  must  have  the  right  person,  at 
the  right  place,  and  at  the  right  time.  Work 
force  reductions  will  continue  to  challenge 
our  young  people  and  overtax  our 
experienced  ones.  IPD  in  this  environment 
will  require  more  emphasis  on  planning  and 
teamwork  and  less  on  empowerment  as  the 
trend  is  to  stick  with  matrix  management. 
Both  up-front  planning  and  the  right 
place/time  place  tenets  dictate  that  a  trained 
IPT  must  be  in  place. 

In  order  to  ensure  that  up-front  planning 
is  conducted,  IPTs  are  assisted  by 
organizations  called  RFP  Support  Offices  at 
AFMC  Centers.  Some  centers  combine 
experts  who  know  IPD,  RFP  documentation 
and  source  selection,  and  technical  analysis 
into  one  staff.  This  staff  assists  SPOs  with 
tools,  documentation,  timelines,  and 
strategy.  It  is  this  independent  organization 
which  ensures  an  integrated  approach  is 
considered  and  applied.  Integration 
(another  tenet)  means  not  only  that  teams  are 


in  place  and  working  together,  but  that  the 
contractor's  plan  to  produce  a  product  is 
integrated  (time  phasing  of  tasks  and  the 
tasks  themselves  among  all  the  functionals) 
throughout  the  total  program  (again,  see 
seamless  management  tools). 

SEAMLESS  MANAGEMENT  TOOLS 

Few  people  in  AFMC  product  centers 
build  any  hardware.  In  a  way,  our  product  is 
paper  such  as  an  RFP,  and  our  customers  are 
the  contractors  who  try  to  figure  out  what 
we  really  want.  In  addition,  we  manage. 
Seamless  management  tools  are  used  for 
both  source  selection  and  management,  and 
that  is  why  they  are  seamless.  Prior  to  IPD, 
seamless  tools  did  not  exist.  One  major 
problem  was  that  the  technical  proposal  of 
the  contractor  were  not  made  part  of  the 
contract.  He  could  promise  the  world,  but  it 
was  the  contract  language  that  was  taken  to 
court,  and  his  proposal  was  generally  not  a 
part  of  it.  When  it  came  to  explaining  his 
understanding  of  a  variety  of  functional 
areas,  little  real  information  was 
forthcoming.  We  needed  a  system  which 
required  the  demonstration  of  the  ability  to 
accomplish  the  work,  not  a  marketing  pitch 
That  is  why  a  major  change  to  the  RFP 
process  was  called  for  and  is  called  the 
integrated  management  framework,  or  IMF. 
Under  this  planning  approach,  there  is  first  a 
simple,  orderly  progression  and  relationship 
from  the  specifications  to  the  work 
breakdown  schedule  (WBS)  to  the  statement 
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of  work  (SOW)  using  a  common  numbering 
system.  In  addition,  this  approach  adds  two 
tools:  the  integrated  master  plan,  or  IMP  and 
the  integrated  master  schedule,  or  IMS.  The 
beauty  of  these  management  tools  is  that  it 
makes  the  contractor  develop,  prior  to 
contract  award,  both  IMP,  (akin  to  the 
SEMS),  IMP  narratives  (related  to  the 
SEMP)  and  an  IMS  (akin  to  the  SEDS). 
See  MiI-Std-499B  for  more  information  on 
SEMP,  SEMS,  and  SEDS. 

The  IMP  is  incorporated  into  the  contract 
as  an  attachment  and  is  based  on  the 
contractors  understanding  of  the  programs 
key  events,  significant  accomplishments  that 
must  be  concluded  in  order  to  hold  an  event, 
and  how  one  determines  if  an 
accomplishment  is  successful.  It  is  a 
management  tool  used  for  both  source 
selection  and  post  award  administration.  The 
reason  this  tool  is  so  effective  is  it  forces  the 
contractor  to  demonstrate  the  actual  plan  for 
the  entire  program.  The  most  common 
complaint  from  contractors  is  that  it  is  too 
expensive  to  build  (particularly  with  no 
assurance  of  winning)  and  requires  to  much 
up  front  work.  This  indicates  that  in  the  past 
many  of  these  contractors  did  not 
accomplish  the  necessary  proposal 
preparation  which  resulted  in  a  propensity  to 
rely  on  ECPs  to  get  well. 

The  IMS  provides  a  detailed  schedule 
(calendar  dates)  to  meet  those  events.  This 
new  IMF  also  calls  for  narrative  summaries. 
These  plans,  which  address  areas  such  as 
configuration,  quality,  manufacturing,  etc.. 


were  previously  provided  by  CDRL  after 
contract  award.  Narratives  are  now 
provided  to  the  source  selection  team  and 
show  not  only  the  key  processes  (as  opposed 
to  them  spitting  back  a  Mil-Std),  but  how 
they  all  integrate. 

TEAMWORK  AND 

COMMUNICATION 

Under  the  old  way  of  doing  business, 
team  players  operated  more  or  less  in  a 
functional  vacuum.  Now  inter-functional 
interaction  is  being  asked  for  where  team 
players  work  both  inside  and  outside  their 
own  particular  area  of  expertise.  Why? 
Because,  when  this  was  done  by  the 
Japanese  in  their  plants  it  was  found  that  the 
gains  of  even  a  slightly  less  proficient  expert 
were  offset  by  the  gains  in  team  problem 
solving  and  product  improvement.  Does 
this  mean  that  by  sitting  and  working  issues 
together  that  one  could  expect  a  better 
product  faster?  The  answer  is  yes.  How  does 
it  work?  It's  quite  simple,  really.  To  expect 
a  functional  expert  to  be  the  only  person 
capable  of  solving  a  problem  effecting  his 
area  is  shortsighted,  but  we  have  been 
operating  that  way  for  years  through  out 
"stovepiped"  relationships.  Team  members 
can  bring  a  variety  of  perspectives  and 
potential  solutions  to  processes  and 
problems  that  they  are  not  the  expert  on. 
This  assumption  is  intuitive.  A  well 
balanced,  motivated  and  capable  team,  with 
competent  leadership  can  produce  products 
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that  far  out  perform  those  of  stovepiped 
teams. 

Teamwork  also  involves  establishing 
proper  relationship  of  the  team  leader  to  the 
workers.  First  of  all,  as  mentioned  earlier,  if 
a  leader  is  going  to  have  the  full  attention  of 
the  members,  the  functionals  must  report  to 
and  have  his  personnel  evaluation  done  by 
the  project  leader,  not  the  functional  head. 
The  functional  head's  job  is  to  provide 
qualified  people  to  the  team,  not  matrix 
manage  (at  least  not  under  the  lean  or  IPD 
culture).  Secondly,  under  lean  practices, 
people  are  assigned  for  the  life  of  the 
project.  Why?--  To  keep  people  in  their  jobs 
where  they  can  make  use  of  all  their 
experience  and  training!  This  points  to 
civilianizing  many  acquisition  jobs.  Thirdly, 
the  team  must  have  goals  and  objectives  and 
as  such,  they  should  be  written  down  in 
some  form  of  a  team  agreement  (charter). 

The  cultural  hurdle  though  which  is  most 
distressing  to  overcome  is  to  get  the  proper 
number  of  people  involved  up  front.  There  is 
a  reluctance  to  staff  an  organization  during 
the  most  critical  phases  (CE  and  Dem  Val). 
It  is  generally  during  this  time  that 
organizations  are  begging  for  resources. 
This  is  not  lean,  but  is  the  old  mass 
production  mentality.  Not  having  the  right 
people  early  on  (right  time)  delays  problem 
solving  and  causes  major  cost  problems  later 
on.  It  is  the  leader's  job  to  encourage  conflict 
among  his  team  when  the  impact  of  change 
is  minimal.  In  order  to  do  this,  he  leads  the 
group  into  an  agreement  on  how  to  handle 


conflicts  and  documents  it.  This  healthy 
conflict  should  result  in  a  number  of 
potential  alternatives  which  will  require 
these  professionals  to  mix  solutions,  risk, 
and  costs.  These  alternatives,  and  resultant 
trade  studies,  are  the  business  of  our  product 
centers  using  the  discipline  provided  in  Mil- 
Std-499B,  and  therefore  center  involvement 
is  mandatory,  particularly  during  Phase  0 
when  the  payoffs  are  so  high.  Team 
composition  in  a  lean  organization  shrinks 
over  time.  This  is  just  the  opposite  of  mass 
production  (or  government  approach)  where 
teams  are  generally  small  and  grow  over 
time.  No  foreseeable  change  is  expected 
within  DoD  unless  we  improve  our  up-front 
planning,  use  seamless  tools,  and  empower 
all  the  players. 

LEAN  ENGINEERING  PRODUCTION 

The  previous  discussion  has  been  on  how 
the  derived  tenets  (from  lean  practices)  of 
IPD  can  be  employed  in  DoD  and  their 
limitations.  But  lean  engineering  also 
involves  a  number  of  other  concerns.  Most 
of  these  have  to  do  with  the  actual 
production  of  the  hardware  and  subcontract 
relationships.  The  successes  of  these 
techniques  have  been  a  major  motivater  for 
incorporating  lean  practices  in  companies 
outside  Japan.  The  focus  also  has  been  high 
volume  manufacturing.  In  DoD,  should  we 
try  to  incorporate  these  principles  into  low 
volume  manufacturing  or  even  non¬ 
manufacturing  "white  collar"  areas,  such  as 
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the  SPO  management  process?  Should  we 
be  attempting  to  make  our  contractors 
implement  lean  practices.  If  so,  what 
influences  do  we  have  in  making  an 
organization  or  a  contractor  lean  and  what 
kind  of  success  can  we  expect? 

What  kind  of  data  can  be  used  to  show 
the  value  of  lean  engineering/production? 
Let's  compare  US  traditional  methods  of 
manufacturing  and  engineering  against  the 
Japanese  lean  practices. 

1)  The  average  car  company  team  size: 
US-1421;  Japan-333. 

2)  The  average  number  of  engineering 
design  hours:  Japan-1.7  million;  US  or 
European  manufacturer-3  million. 

3)  The  amount  of  time  it  takes  to  field  a  car: 
Japan-46  months;  US  or  Europe-60  months. 
How  is  this  possible  and  more  importantly, 
can  we  expect  similar  improvements  to 
show  up  in  government  acquisition? 

In  preparing  the  RFP,  all  the  IPD  tenets 
are  applicable  and  can  result  in  huge  savings 
in  managing  programs.  However,  in 
achieving  the  type  of  savings  listed  above, 
those  savings  must  be  made  by  your  prime 
contractor.  These  are  manufacturing  or 
engineering  savings.  What  can  the  program 
manager  or  the  government  do? 

One  of  the  most  critical  issues  in 
achieving  these  savings  is  in  your 
contractor's  relationship  with  its  first  tier 
suppliers.  How  are  Japanese  companies  so 
different  from  their  US  or  European 
counterparts,  and  secondly,  what  are  the 
government's  limitations  in  implementing 


these  ideas?  Supplier  (or  subcontractor) 
tenets  may  be  laid  out  into  four  major 
categories.  As  we  go  through  them,  think  of 
how  DoD  could  use  them  to  make  a 
difference. 

SUBCONTRACTING  RULES 

What  should  be  the  role  of  the 
government  in  order  to  be  a  lean 
organization?  Do  our  rules  on 
subcontracting  result  in  lower  prices,  higher 
reliability,  or  better  quality?  Whether  a 
contract  is  anticipated  to  be  sole  source  or 
competitively  awarded,  the  contractor  IAW 
FAR  Part  19  and /  or  52,  is  required  to  follow 
certain  clauses  (and  to  administer  them).  The 
sheer  number  of  the  clauses  effecting  the 
relationship  between  the  prime  and  the  sub 
is  staggering  and  totals  around  100.  These 
clauses  effect  areas  such  as  quality, 
inspection,  warranty,  sources,  management, 
security,  examination  of  records,  audit,  use 
of  SB/SDB,  EEO,  patents,  CAS, 
competition,  incentives,  rights,  data, 
markings,  and  proprietary  rules.  What 
impact  do  all  the  clauses  have  and  how  is  it 
different  from  lean  production?  The 
contention  could  be  made  that  ensuring  all 
these  clauses  are  followed  does  not  add 
anything  to  the  product,  but  it  is  expensive. 
None  of  these  clauses  either  by  themselves 
or  together  has  anything  to  do  with  being 
lean.  For  the  most  part,  they  are  purely 
administrative  and  increase  the  indirect  cost 
of  a  part.  Could  they  all  be  removed  from 
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the  books  with  no  impact  to  the  quality  or 
the  cost  of  the  product?  This  is  an  example 
of  the  government's  micro-management 
approach  which  burdens  the  customer  and 
does  not  place  responsibility  or  real 
penalties  for  non-performance  on  the 
contractor.  The  government  is  focused  less 
on  quality  or  reliability  and  more  on  price 
and  social  programs.  Our  procedures  are  the 
antithesis  of  a  lean  organization.  If  the  rules 
must  remain,  remove  the  need  to  check  on 
them.  Trust  is  lean. 

TEAMWORK  AND  TRUST 

In  industry  as  well  as  government, 
simple  parts  will  have  multiple  suppliers. 
Complex  parts  may  have  only  one.  There  is 
a  flawed  perception  that  developing  a 
second  source  is  the  only  way  to  protect  the 
government  from  overpricing.  This  is  not 
lean  thinking  because  pricing  and  cost 
considerations  are  based  on  a  mistrust  of  the 
industry,  not  on  teamwork  or  long  term 
relationships.  The  concepts  of  teamwork  and 
trust  are  lean.  Subcontractor  relationships  to 
the  prime  are  considerably  different  in  a  lean 
production  culture.  In  cases  the  prime  holds 
stock  in  many  of  its  first  tier  suppliers.  In 
fact,  it  is  common  to  see  mid  level  managers 
of  the  prime  being  assigned  into  the 
management  structure  of  one  of  it  suppliers. 
But  the  real  key  to  effective  supplier 
relationships  starts  with  the  contract  between 
the  two  entities.  In  the  US,  the  basis  for 
award  is  price.  The  automotive  industry  also 


has  its  share  of  "buy-ins."  The  primes  know 
this  and  actually  estimate  the  rate  at  which 
parts  can  be  expected  to  rise  in  price. 
Contracts  are  generally  for  one  year  with 
options.  The  amount  of  trust  is  minimal  as 
numerous  times  subs  have  lost  the  ability  to 
get  back  their  tooling  or  investment  costs  on 
multi-year  option  contracts  when  the  prime 
goes  out  at  the  end  of  the  first  year  with 
either  a  new  invitation  for  bid  (IFB)  or 
decides  to  do  it  in-house. 

In  the  government,  subcontractor  prices 
are  either  1)  competitively  determined  by 
the  prime,  2)  the  contract  is  not  subject  to 
audit  due  to  a  determination  of  "adequate 
price  determination",  or  3)  subject  to  a 
DCAA  audit  for  determining  cost.  The  point 
to  be  made  here  is  that  the  cost  of  an  item  is 
not  subject  to  any  type  of  "value  analysis". 
Value  analysis  is  the  continuous 
identification  of  critical  costs  and  steps  with 
a  goal  of  incrementally  improving  them.  The 
profit  of  an  item  may  be  low  (and  the 
government  goes  to  great  length  to  insure 
this,  i.e.,  fair  and  reasonable),  but  the  total 
price  is  still  quite  high  because  the  elements 
of  costs  are  not  targeted  for  improvements. 
Did  the  government  "do  good"?  Many 
would  say  yes  unfortunately.  The  goal  of  the 
prime  should  be  to  minimize  the  price  while 
maintaining  a  reasonable  profit.  This  does 
not  occur  in  the  government.  How  does  the 
lean  producer  work? 

First  of  all,  the  basis  for  award  is  not 
price.  It  is  reliability  and  performance.  Next, 
the  prime  and  sub  jointly  determine  the 
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initial  price  through  the  sharing  of  technical 
and  cost  data.  That  does  not  occur  today  in 
the  government  or  in  most  US  companies. 
Next,  this  partnership  looks  at  ways  to 
reduce  the  cost  of  the  part  through  "value 
analysis,"  and  the  prime  incentivizes  the  sub 
through  value  engineering  sharing 
relationships.  The  prime  expects  the  costs  to 
go  down  over  time,  not  up  as  in  the  US 
model.  The  basis  of  this  relationship  is  the 
primes  long  term  contractual  commitment  to 
the  sub.  The  sub  shares  information  with  the 
prime  not  only  in  the  cost  area  but  in  the 
technical  area  as  well.  It  doesn't  hide  it  for 
the  sake  of  future  negotiations.  Sometimes 
the  prime  might  employ  two  subs  in  a 
sharing  of  orders.  However,  the  basis  for 
future  splits  is  not  cost,  it  is  again  reliability. 
The  reason  it  can  do  this  is  because  the 
foundation  for  future  pricing  is  one  of 
teamwork,  trust,  and  an  understanding  that 
the  prices  will  go  down,  not  up.  Can  DoD 
primes  do  this  on  individual  contracts  — 
probably  only  in  a  few  cases  due  primarily 
to  the  relatively  low  quantity  orders.  The 
trust  through  longer  contracts  and  team 
interaction,  when  coupled  with  DOD’s 
budget  instability,  results  in  these  types  of 
relationships  being  more  an  exception  to  the 
rule  than  the  rule. 

INTEGRATION 

The  next  area  has  to  do  with  the 
"amount"  of  integration,  which  is  another 
success  story  for  lean  production  when 


compared  to  its  US  counterpart.  US 
manufacturers  will,  for  example,  order  all 
the  parts  that  make  up  a  car  seat,  and  then 
assemble  it  at  the  plant.  Japanese  asks  the 
sub  to  not  only  assemble  the  seat,  but  to 
design  it  as  well.  What  does  this  mean?  It 
means  that  not  only  is  the  prime  not 
interested  in  all  detailed  design,  but  the 
number  of  parts  ordered  by  the  prime  is 
reduced,  and  with  it  all  the  indirect  labor  that 
goes  with  both.  Lean  producers  add  only 
27%  value  versus  70%  for  GM.  In  addition, 
only  30%  of  the  part  are  detailed  designed 
by  the  prime  versus  81%  for  GM.  What  this 
results  in  is  that  GM  models  have  3-8  times 
the  number  of  suppliers  as  Toyota,  and 
roughly  6000  people  in  purchasing  versus 
337  for  Toyota.  How  could  this  be  applied 
to  DoD  contracts?  Can  you  as  a  program 
manager  get  your  DoD  prime  to  somehow 
increase  their  sub's  portion  of  the  design  and 
reduce  the  number  of  parts  delivered?  Do 
you  really  care,  particularly  if  it  is  a 
competitive  environment?  It  is  very  doubtful 
whether  or  not  the  government  would  want 
to  get  involved  in  the  breakout  of  parts  and 
design.  Component  breakout  is  manpower 
intensive  and  is  less  and  less  attractive. 
Directed  subcontracting  shifts  the  burden  of 
failure  to  the  government.  Savings  in  this 
area  appear  marginal  in  government 
contracts. 

DELIVERY  AND  INVENTORY 
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The  last  area  has  to  do  with  delivery. 
Lean  producers  utilize  just  in  time  (JIT).  In 
1983,  70%  of  US  suppliers  delivered  once  a 
week.  In  Japan  roughly  16%  deliver  once  a 
week.  So  what's  the  difference?  Looks  like 
the  US  is  doing  pretty  good!  First  of  all,  the 
other  84%  in  Japan’s  case  are  delivering  on 
daily  and  sometimes  hourly  rates,  so  the 
statistics  are  misleading  .  The  more 
fundamental  question  might  be  "what  is  the 
big  deal  with  JIT"?  It  seems  to  add  a  high 
amount  of  risk  if  supplies  don't  arrive  and 
the  line  shuts  down!  JIT  does  two  things. 
First  of  all  it  mandates  higher  quality 
standards  exactly  for  the  aforementioned 
possibility,  a  shut  down  of  the  line. 
Remember,  quality  requirements  as  well  as 
delivery  are  part  of  the  prime  and 
subcontract  arrangement.  In  the  US,  quality 
is  determined  generally  by  a  percentage  of 
bad  parts  in  a  lot.  If  it's  higher  than  allowed, 
the  entire  lot  is  rejected.  In  lean,  because  the 
delivery  schedules  are  so  frequent,  bad  lots 
are  smaller,  and  fixes  can  occur  much 
sooner. 

In  addition,  the  cost  of  holding  large 
inventories  is  an  expense.  Neither  the  prime 
nor  the  sub  wants  this.  So  one  might  expect 
to  get  higher  quality  at  lower  cost  with  a  JIT 
system  and  that  is  the  impetus  for  so  many 
US  companies  moving  in  this  direction.  But 
the  statistics  are  misleading  for  another 
reason.  Although  the  weekly  delivery  rates 
have  improved,  US  primes  are  still  requiring 
subs  to  hold  inventories  and  subs  are 
continuing  to  build  in  large  lots.  More 


frequent  deliveries  are  not  due  to  any  real 
lean  production  techniques. 

Can  you  as  the  program  manager  effect 
delivery  schedules  of  subs?  Again,  the 
question  must  be  asked,  do  you  even  want  to 
get  involved?  What  kind  of  risk  does  JIT 
add  to  your  schedule?  For  low  quantities,  is 
it  even  desirable?  JIT  has  been  a 
fundamental  reason  for  Japanese  successes, 
but  it  applicability  to  government  contracts 
appears  minimal. 

RECOMMENDATIONS 

Based  on  the  discussion  above,  what 
should  DoD  be  focusing  on  in  terms  of 
becoming  a  lean  organization?  If  IPD  is  a 
desirable  management  philosophy,  we  need 
people  who  understand  its  limitations  and 
applications,  particularly  in  view  of  future 
massive  manpower  cuts.  Here  are  some 
recommendations,  first  in  the  area  of  IPD 
tenets,  and  secondly  in  production  and 
supply. 

TENETS 

Managers  at  the  senior  level  must  address 
the  paradigm  shift  required  of  empowerment 
for  both  its  in-service  employees  and 
contractors.  Decision  making  must  be 
decentralized.  IPD  offices  at  headquarters 
and  at  the  centers  should  be  brainstorming 
areas  where  manpower  could  be  saved  by 
empowering  its  people.  To  move  from 
"stovepiped"  relationships  that  are  more 
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concerned  with  process  to  true  IPTs  teams 
focused  on  products,  IPT  leaders  need  to  be 
empowered  with  real  authority  for  members, 
and  teams  need  to  be  trained  as  teams  with 
their  customer.  Little  training  exists  today. 
RFP  support  offices  are  not  being  fully 
utilized  to  ensure  up  front  planning  is  a 
disciplined  process.  This  needs  to  be  beefed 
up  and  should  be  reviewed  by  all  services  to 
see  if  it  has  wider  application. 

Seamless  management  tools  and  their  use 
in  DoD  appear  to  be  moving  slowly.  The 
reason  for  this  most  likely  is  that  few  people 
understand  them  (IMP/IMS).  These  tools  as 
pointed  out  work  with  a  number  of  the  other 
tenets  to  improve  the  chance  of  a  quality 
product.  The  problem  is  few  people  are 
willing  to  change,  or  understand,  how  these 
tools  work.  If  these  are  the  tools  that  make 
sense  to  use,  lets  get  behind  them  and  train 
people  on  their  use. 

Teamwork  is  absolutely  essential 
between  not  only  center  functionals,  but 
with  our  customers  and  contractors  as  well. 
Training  the  IPT  as  a  group  is  a  must.  More 
problems  with  satisfying  the  customer  come 
from  the  inability  to  understand  what  is 
really  wanted  than  anything  else.  A  charter 
laying  out  these  understandings  is  one 
method  to  ensure  communication,  and  roles 
are  clear.  A  charter  is  recommended  for 
every  IPT  which  addresses  such  items  as 
mission,  schedule,  documentation,  roles, 
products,  processes,  and  decision  making 
rules. 


PRODUCTION/SUPPLY 

Finally,  what  can  DoD  do  in  the  area  of 
supplier  relationships  to  reduce  the  cost  of 
business?  First,  there  are  a  number  of 
streamlining  initiatives  at  high  levels 
looking  at  ways  of  cutting  down  on  the 
number  of  laws  effecting  acquisition.  This  is 
positive  step  if  and  only  if  some  tangible 
action  results.  Past  initiatives  have  resulted 
in  great  ideas  but  no  change.  DoD  needs  to 
look  at  the  laws  effecting  supplier 
relationships  specifically  to  see  what 
changes  can  effect  quality,  cost,  and 
reliability. 

How  do  we  incentivize  teamwork  and 
trust?  It  does  appear  that  the  idea  that  a  sub 
would  ever  trust  its  prices  or  technology  to 
the  prime  is  un-American.  The  only  reason 
it  is  done  in  Japan  is  that  the  prime  owns 
part  of  the  sub,  so  there  is  a  mutual  benefit. 

How  can  we  institutionalize  "value 
analysis"?  In  the  areas  of  integration  can  we 
incentivize  the  reduction  of  engineering 
design  hours  and  piece  part  build-up  at  the 
prime's  location?  How?  Why  do  it? 

What  are  the  risks  associated  with  JIT 
delivery  systems,  particularly  when  parts  are 
extremely  expensive  and  complex?  Is  it  even 
a  player  in  low  quantity  manufacturing? 

CONCLUSION 

IPD  implementation  is  moving  very 
slowly  for  two  reasons.  Few  people 
understand  the  process  and  there  are  few 
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people  who  can  teach  them.  Another  major 
issue  is  change,  and  people's  natural 
reluctance  to  try  something  new.  IPD  works. 
Industry  is  employing  it.  But  the  people  in 
DoD  who  can  impact  change  have  been 
working  for  so  many  years  under  one  set  of 
rules  that  to  learn  a  new  technique  is 
difficult.  In  order  for  DoD  to  accomplish  its 
mission,  new  ways  of  doing  business  must 
be  created.  Manpower  reductions  alone 
dictate  this.  IPD  requires  managers  to  face  a 
fundamental  paradigm  shift,  and  those  who 
can't,  or  won't  face  it,  are  going  to  find 
success  difficult.  It  is  for  this  reason  that  a 
cultural  change  is  required.  IPD  if  not  the 
whole  answer,  is  certainly  a  step  in  the  right 
direction. 

There  continues  today  to  be  many 
questions  on  what  to  expect  out  of  this  new 
philosophy  of  IPD.  Production  savings  and 
management  applications  appear  difficult 
and  hard  to  find.  The  purpose  of  this  paper 
has  been  to  get  you  thinking! 
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Abstract 

The  U.S.  Federal  Government  significantly 
underrewards  investment  in  quality  in  its 
software  procurements.  This  paper  presents 
a  formal  theory  for  why  the  underinvestment 
in  quality  occurs.  The  theory  stresses 
differences  in  the  measurability  of 
dimensions  of  performance  and  the 
dysfunctional  incentives  for  government  and 
industry  managers  from  too  great  an 
emphasis  on  the  measurable.  We  present 
evidence  that  software  process  improvement 
(SPI)  is  a  means  for  increasing  attention  to 
quality  in  government  software  procurement. 
Based  on  our  findings,  we  conclude  that:  1) 
the  government  should  consider  incentives 
for  SPI,  2)  reduce  reliance  on  incentives  for 
cost  and  schedule  performance,  and  3)  ensure 
the  government  fields  reliable  methods  and 
effective  teams  for  evaluating  contractors' 
SPI  efforts. 

1.0  Introduction 

The  U.S.  Federal  Government  has  had  grave 
difficulty  acquiring  software,  an  increasingly 
important  component  of  both  military  and 
civilian  systems.  Most  software  acquisitions 
take  much  longer  than  planned  and  cost  a  lot 
more  than  budgeted.  Acquired  systems  do 
not  work  as  planned,  if  they  work  at  all. 
Perhaps  General  Bernard  Randolph, 
commander  of  the  U.S.  Air  Force's  system 
procurement  organization,  summarized  the 
government's  problem  best  when  he  opened 
a  1989  international  software  conference  by 
stating  that,  "The  Air  Force  has  a  perfect 
record  when  it  comes  to  software 
procurement,  we've  been  late  and  over 


budget  on  every  program.  "1  The  record  on 
software  quality  is  almost  as  perfect:  few,  if 
any,  systems  accepted  and  paid  in  full  can 
fulfill  their  intended  functions  without 
considerable  rework. 

Software  procurement  successes  are 
extremely  rare  for  systems  of  any 
consequential  size  and  complexity.  Software 
procurement  disasters,  systems  that  cost  too 
much,  take  too  long,  and  do  not  work  as 
intended,  are,  however,  common:  the 
Federal  Aviation  Administration's  new  air 
traffic  control  system,  the  Internal  Revenue 
Service's  processing  systems,  the  U.S.  Air 
Force's  Peace  Shield  program  for  Saudi 
Arabia,  the  Department  of  Commerce's  next 
generation  weather  radar,  either  the  original 
or  the  current  upgrade  to  the  Department  of 
Defense's  (DoD)  Cheyenne  Mountain 
complex,  and  the  Department  of  Commerce's 
new  patent  administration  system  are  but  a 
few  example  disasters.  Even  when  success 
is  declared  at  some  point  in  the  course  of 
development  and  implementation,  latent 
defects  may  strike  later  converting  an 
apparent  success  to  a  dismal  failure;  e.g.,  the 
AT&T  phone  switch  or  Clementine 
spacecraft.  ^ 

The  sources  of  the  government's  difficulties 
in  software  acquisition  are  extremely  complex 
and  much  studied.  There  are  an  extensive 
academic  literature  and  the  reports  of  many 
commissions  and  organizations  on 
government  acquisition  (see,  for  example,  [2- 
9]).  Many  of  the  reforms  recommended  in  all 
of  this  prior  work  have  not  been 
implemented.  Those  reforms  that  have  been 
implemented  have  not  demonstrably 
improved  the  software  acquisition  process. 
Some  reforms,  such  as  placing  software 
standards  into  contracts  across-the-board, 
have  actually  been  detrimental  by  not 
accommodating  industry's  evolving  best 
practices,  forcing  the  Defense  Department 


*  Keynote  Speech,  1989  Tri-Ada  Conference, 
Pittsburgh,  PA,  October. 

2See  [1]  for  a  more  extensive  discussion  of  the 
difficulties  the  government  has  acquiring  large 
software  systems. 
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into  yet  another  revision  of  their  software 
procurement  standards. 

The  roots  of  the  government's  difficulties  in 
software  acquisition  are  in  dysfunctional 
incentives  for  both  government  procurement 
managers  and  software  contractors.  Several 
factors  make  the  incentives  dysfunctional. 
One  major  contributing  factor,  particularly  in 
defense  software  acquisition,  is  the  regular 
separation  of  software  contracts  into 
development  and  maintenance  phases  with 
different  groups  doing  each  phase. 
Developers  know  that  they  will  not  have  to 
live  with  the  consequences  of  any  quality 
problems  with  the  software  they  develop 
because  it  will  be  some  other  group's  task  to 
make  the  software  operational  and  to  maintain 
it.  The  developer's  main  objective  is  to  get 
the  government  agency  to  accept  and  pay  for 
whatever  it  is  that  they  develop.  Developers 
enjoy  further  protection  from  the  negative 
consequences  of  producing  poor  quality 
software  because  the  government  makes 
inconsistent  and  infrequent  use  of  past 
performance  information  in  the  selection  of 
firms  for  new  software  development  projects. 
Past  performance,  good  or  bad,  simply  does 
not  matter  very  much  to  your  prospects  for 
getting  new  software  development  business 
from  the  government. 

Another  important  contributing  factor  to 
dysfunctional  incentives  and  the  primary 
focus  of  this  paper  is  differences  in  the 
measurability  of  dimensions  of  performance 
in  writing  and  enforcing  contract  provisions. 
Cost  and  schedule  are  nominally  easy  to 
measure.  For  the  government  customer, 
software  quality  is  difficult,  often  impossible, 
to  measure  prior  to  extensive  operational 
experience  with  the  system.  There  is 
considerable  oversight^  pressure  in 
government  acquisition  where  the  "public's 
money"  is  being  spent  to  emphasize  the 
measurable  dimensions  of  performance 
throughout  the  contracting  process.  Software 
quality  often  gets  short  shrift  because 
individual  careers  are  much  better  served  by 


^ Approximately  40  percent  of  every  federal 
procurement  dollar  goes  to  management  oversight. 
Five  to  fifteen  percent  is  typically  allocated  for 
management  oversight  in  the  commercial  sector  [9]. 


association  with  projects  that  may  produce 
poor  quality  software  but  that  are  on  schedule 
and  under  budget  than  by  association  with 
projects  producing  great  quality  software 
over  budget  and  behind  schedule. 

This  paper  utilizes  a  "principal-agent  model' '4 
to  examine  government  and  contractors' 
incentives  in  software  procurement.  We 
provide  a  theoretical  explanation  for  why 
existing  incentives  in  software  acquisition  are 
detrimental  to  software  quality.  We  also  use 
the  model  to  explore  how  the  government  can 
provide  indirect  incentives  for  better  software 
quality.  The  model  explores  incentives  in  a 
process  with  two  easily  measured  dimensions 
of  performance,  cost  and  schedule,  and  one 
difficult  to  measure  dimension,  quality.  We 
extend  the  model  to  consider  software 
process  improvement  (SPI)  as  an  approach 
for  improving  incentives  in  software 
procurements.  We  close  with  several 
recommendations  for  software  procurement 
reform. 

2.0  Incentives  in  Software 
Acquisition 

The  incentives  for  government  and  industry 
managers  center  primarily  around  issues  of 
cost  and  schedule  performance.  Glenford 
Myers,  one  of  the  early  researchers  to 
document  the  importance  of  software  quality, 
asserted  "We  try  to  solve  the  problem  by 
rushing  through  the  design  process  so  that 
enough  time  is  left  at  the  end  of  the  project  to 
uncover  the  errors  that  were  made  because 
we  rushed  through  the  design  process"  [17]. 
Myers  is  alluding  to  a  common  industry 
practice  of  rushing  through  software 
development  in  order  maximize  revenue,  with 
the  end  of  the  project  left  for  the  firm  to 
"build-in"  quality  until  the  government  takes 
acceptance.  As  an  illustration,  in  the  opening 
statement  of  a  recent  program  review  meeting 
for  a  multi-billion  dollar,  software-intensive 


^The  Principal-agent  framework  is  very  flexible  and 
has  been  used  to  study  risk  allocation  in  share 
cropping  [10],  insurance  markets  [11],  education  [12], 
and  measurement  in  organizations  [13].  Several 
researchers  have  used  this  framework  to  study  general 
government  acquisition  [14-15]  and  software 
development  for  internal  use  in  large  commercial 
organizations  [16]. 
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government  procurement,  the  president  of  the 
contracting  firm  told  his  customers  that  "the 
software  is  done,  we're  just  trying  to  get  it  to 
work. 

Government  managers  are  evaluated 
primarily  on  their  ability  to  contain  costs  and 
meet  schedules.  Most  of  their  training 
involves  understanding  and  constructing 
budgets  and  schedules,  attributes  of 
procurement  that  are  easy  to  quantify  and 
understand.  Explicit  cost  and  schedule 
incentives  are  the  rule  in  contracts.  Explicit 
contractual  incentives  for  software  quality, 
attributes  such  as  reliability,  maintainability, 
or  number  of  defects  per  module  of  software, 
are  rare.  Government  and  industry  managers 
attend  primarily  to  explicit  contractual 
provisions.  Software  quality  is  never  very 
explicit  and  never  gets  much  attention. 

The  incentives  for  cost  and  schedule 
performance  are  usually  found  in  contract 
clauses  or  award  fee  pools.  Sometimes, 
additional  profit  is  tied  to  meeting  schedule  or 
cost  targets  or  achieving  superior  system 
performance.  Abstractly,  industry  receives 
progress  payments  by  expending  effort  on 
specified  tasks  in  their  contract.  As  the  actual 
cost  of  a  task  rises  toward  the  task's 
budgeted  cost,  the  government  responds  with 
payments  up  to  approximately  80  to  90 
percent  of  the  budgeted  cost.  These 
payments  arrive  even  though  the  actual  cost 
of  work  performed  may  have  no  relation  to 
whether  a  task  is  25,  50,  or  80  percent 
complete.  This  is  especially  critical  during 
the  early  phases  of  a  software  development 
when  rigorous  measures  of  outcome  or 
quality  are  virtually  non-existent. 

A  great  deal  of  pressure  is  applied  by 
industry  and  Congress  to  reach  schedule 
milestones  as  soon  as  possible.  Most 
government  managers  are  only  too  happy  to 
accommodate  because  their  evaluations  also 
hinge  on  accomplishing  these  milestones. 
Further  pressure  may  come  from  Congress  in 


^Private  communication  between  one  of  the  authors 
and  the  government  contracts  manager  for  the  affected 
program  during  the  summer  of  1993.  The  referenced 
executive  program  review  took  place  on  March  2, 
1993. 


the  form  of  the  dysfunctional  practice  of 
cutting  the  budgets  of  programs  that  are  late 
or  over  budget.  This  irrational  behavior 
almost  always  drives  up  the  life  cycle  cost  of 
a  system. 

A  reason  for  much  of  the  obsession  for  cost 
and  schedule  performance  lies  with  the 
infeasible  budgets  and  schedules  that  the 
political  system  forces  on  government  and 
industry  managers.  It  is  such  a  major 
problem  that,  at  this  writing,  a  major  study  is 
underway  within  the  Office  of  the  Secretary 
of  Defense  to  document  the  widespread  cost 
growth  of  programs  within  the  DoD.  A 
preliminary  version  of  the  study  found  that 
"for  over  several  hundred  procurements,  in 
almost  every  case,  [actual]  costs  in  each 
program  as  they  have  matured  grossly 
exceeded  the  projected  costs"  [18]. 

Cost  growth,  a  problem  plaguing  government 
procurement  throughout  our  history,  occurs 
as  a  result  of  a  combination  of  factors: 

•The  political  system  fails  to  fully  fund 
programs,  thus  stretching  out  their 
development  life  and  cost; 

•The  agencies  deliberately 
underestimate  costs  so  that  Congress 
will  initially  fund  a  project,  thereby 
establishing  a  powerful  constituency 
for  the  project; 

•Software-intensive  contracts  are 
awarded  to  inexperienced  firms;  and 

•There  are  major  technological 
uncertainties,  particularly  in  defense 
procurements,  as  government 
agencies  often  pursue  aggressive 
performance  targets  while  paying  little 
attention  to  issues  of  cost  containment 
[3-4]. 

Government  managers  procuring  software 
routinely  work  with  "shortened"  schedules 
and  "reduced"  initial  budgets  to  meet 
requirements  that  are  sometimes  technically 
unattainable  given  the  funding  and  schedule 
constraints.  The  oversight  for  government 
managers  doing  acquisition  is  oppressive 
with  layer  upon  layer  of  investigators  and 
controllers.  The  dominant  incentive  is  to 
meet  the  measurable  performance  standards, 
not  to  solve  the  government's  fundamental 
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acquisition  objective  of  achieving  quality 
software,  software  that  will  accomplish  the 
intended  government  functions,  at  a 
reasonable  cost  in  a  reasonable  length  of 
time. 

Dysfunctional  incentives  for  firms  also  arise 
from  government  procurement  officials' 
inability  to  use  past  performance  information 
rigorously  when  selecting  firms  for  new 
developments.  Unlike  the  commercial  sector, 
where  consumers  (firms  or  individuals) 
routinely  consider  how  well  a  firm  performed 
in  the  past  as  a  guide  to  how  they  might 
perform  in  the  future,  government  managers 
are  severely  constrained  in  using  past 
performance  information  about  a  firm  when 
making  a  procurement  decision.  Firms  know 
their  inattention  to  software  quality  in  one 
development  will  probably  not  hurt  them 
when  bidding  for  future  procurements.  So, 
the  standard  operating  procedure  within  many 
firms  is  to  rush  through  development  and 
hope  that  the  integration  of  hundreds  of 
software  modules  into  a  system  will  yield  a 
level  of  quality  that  the  government  will 
accept.  This  problem  snowballs  as  most 
government  managers,  facing  significant 
schedule  and  financial  pressure,  become 
willing  accomplices  to  the  procurement  of 
poor  quality  software  by  accepting  software 
products  much  too  soon. 

•  How  The  Incentives  Affect 
Software  Quality:  Government 

procurements  are  separated  into  software 
development  and  support.  Different 
managers  are  responsible  for  each  phase  and 
each  is  graded  only  on  their  phase.  This  is 
especially  significant  for  software 
development  procurements  because  a 
significant  part  of  a  software  system's  life 
cycle  cost  depends  upon  the  quality  of  the 
products  from  the  development  phased 

Inattention  to  software  quality  is  a  cumulative 
problem.  Government  managers  are 
evaluated  on  cost  and  schedule  performance 
for  development,  not  for  the  system's  full  life 
cycle.  So  during  the  development  phase, 


^ A  system's  life  cycle  cost  includes  all  research  and 
development,  production,  and  software  support. 


managers  are  motivated  to  cut  corners  and 
take  delivery  of  a  system  at  the  earliest 
possible  date,  even  though  corners  cut  may 
end  up  costing  the  government  10  to  100 
times  as  much  over  the  life  cycle  of  system 
[19-22],  The  earlier  a  defect  is  introduced  in 
a  development  and  the  longer  it  is  allowed  to 
persist,  the  more  costly  it  is  on  average  to 
remove  when  it  is  found. 7  Poor  software 
quality  is  very  expensive;  at  least  40  percent 
of  a  system's  cost  is  spent  on  system  support 
after  the  system  is  delivered  to  the  user 8. 
Estimates  of  how  much  system  support  is 
allocated  to  maintenance  range  from  17  to 
100  percent. ^  The  quality  of  the  software 
affects  how  much  of  that  software  support  is 
maintenance.  The  lower  the  quality,  the 
greater  the  maintenance.  Thus,  managers  that 
accept  poor  quality  software  during 
development  are  pushing  defect  identification 
and  removal  into  the  system's  support  phase, 
where  defects  are  usually  much  more 
expensive  to  remove. 

An  emerging  approach  for  overcoming  the 
lack  of  management  attention  to  software 
quality  uses  "measures  of  Software  Process 
Improvement"  as  an  easily  measured  proxy 
for  software  quality.  These  measures  reflect 
the  rigor  of  an  organization's  software 


^There  are  many  reasons  for  the  high  expense  of 
maintenance,  but  the  re-engineering  that  often  goes 
on  after  delivery  is  viewed  as  the  main  culprit.  In 
most  cases,  key  developers  and  designers  do  not  stay 
around  for  the  support  phase  of  a  system.  They  move 
on  to  new  challenges,  for  they  are  usually  the 
developing  organization's  most  important  assets. 
Consequently,  personnel  performing  software  support 
are  often  not  only  new  to  a  system,  but  they  also 
usually  work  with  documentation  that  does  not  match 
the  current  system  or  undocumented  software.  In 
personnel  terms,  it  is  much  like  comparing  the 
knowledge  of  an  auto  mechanic  with  that  of  the 
engineer  who  designed  and  developed  your  car. 

^See,  for  instance,  [20,  21,  24-29]. 

^Some  sources  are  [26],  17  percent;  [20],  100 
percent;  and  [29],  63  percent.  The  problem  with 
these  data  and  collecting  such  data  in  general  is  the 
lack  of  uniformity  across  industry  on  what  constitutes 
maintenance.  The  data  provided  by  Lientz  et  al.  do 
not  include  such  items  as  improving  documentation, 
recoding  for  efficiency,  or  environmental  changes 
under  the  maintenance  heading  while  other  researchers 
would  consider  those  maintenance  activities. 
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processes  through  issues  such  as,  "Do  they 
provide  training  for  their  developers  and 
managers?"  and  "Do  they  measure  the 
performance  of  their  processes  and  products 
as  a  normal  business  practice?" 

3.0  Public  Policy  Options 

The  problems  in  acquiring  quality  software 
products  are  so  obvious  and  well-understood 
that  direct  solutions  that  in  one  way  or 
another  reorient  a  firm's  incentives  are 
inviting.  These  solutions  include  holding 
firms  accountable,  awarding  life  cycle 
contracts,  or  restructuring  the  basis  of 
payments  for  firms  by  employing  a 
measurable  proxy  for  software  quality. 

•  Hold  Firms  Accountable:  The 

most  direct  solution  is  to  merely  require  that 
firms  deliver  high  quality  or  we  will  take  our 
business  elsewhere,  in  much  the  same 
manner  as  we  behave  as  consumers.  If  your 
sink  is  leaking  and  you  hire  a  contractor  to  fix 
the  problem,  you  may  have  to  pay  50  percent 
up  front  and  the  remainder  once  you  see  that 
your  sink  is  no  longer  leaking.  With 
software,  however,  work  that  appears 
completed  today  usually  contains  latent 
defects  that  strike  a  system  in  the  future  under 
certain,  typically  unforseeable  conditions. 
Another  difficulty  is  establishing  that  the 
quality  problems  stem  from  "shirking"  rather 
than  from  some  government  error;  e.g., 
incorrect  requirement  specification  or  poorly 
trained  government  oversight  personnel. 
Even  if  past  shirking  is  established,  it  is  not 
always  easy  to  show  that  past  performance  is 
sufficiently  predictive  to  warrant  excluding  a 
firm  from  competing  for  new  business. 
Politics  also  make  it  difficult  to  exclude 
firms.  Many  firms  have  major  constituencies 
in  Congress;  lobbyists  ensure  that  the 
respective  representatives  know  the  numbers 
of  direct  and  indirect  jobs  flowing  from  each 
procurement.  On  the  legal  side,  the  difficulty 
in  firing  or  refusing  to  hire  a  firm  is 
exacerbated  by  the  legal  encumbrances  facing 
the  government  manager  even  if  they  have 
performed  poorly  in  the  past.  10  Although 


past  performance  may  by  law  be  considered 
when  determining  a  contract  award,  most 
implementing  procedures  found  within  the 
government’s  many  procurement  agencies 
have  severely  narrowed  the  use  of  this 
information.  Most  government  procurement 
officials  are  so  apprehensive  about  using  past 
performance  in  source  selections  that  almost  a 
dozen  industry  groups  recently  wrote  a  letter 
to  Senator  Sam  Nunn  requesting  help  [30], 
The  groups  call  on  the  government,  as  part  of 
the  on-going  procurement  reform  initiatives, 
to  make  past  performance  a  real  consideration 
in  future  source  selections: 


Past  performance  is  one  of  the  most  critical 
factors  in  successful  source  selection. 
Current  federal  practice  is  lacking  in 
uniformity  and  consistency  and,  in  too  many 
cases,  is  non-existent.  In  a  procurement  era 
increasingly  characterized  by  complexity, 
high  technology  and  developmental 
performance  demands,  an  assessment  of  a 
contractor’s  relevant  past  performance  is  one 
of  the  strongest  quality  discriminators 
available  to  the  federal  buyer  [30]. 


•  Award  Life  Cycle  Software 
Contracts:  Another  direct  solution  would 
be  to  eliminate  the  separation  between 
development  and  support  of  software 
systems.  Under  these  circumstances,  a 
contract  would  be  awarded  to  a  firm  to 
develop  and  then  support  the  software.  The 
contract  identify  the  level  of  support  the 
government  would  tolerate,  forcing  the  firm 
to  achieve  a  level  of  quality  during 
development  that  would  allow  them  to  realize 
a  profit  during  development.  One  problem 
with  this  approach  is  that  Congress  restricts 
the  length  of  government  contracts  usually  to 
between  one  and  five  years,  depending  upon 
the  nature  of  development.  A  life  cycle 
contract,  encompassing  the  development  and 
support  phase,  could  reasonably  span  5  to  15 
years  for  many  projects.  Even  if  life  cycle 
contracts  were  permitted  within  the  next  few 
years,  it  would  take  years  of  experience  and 
data  collection  before  one  would  know  how 
to  structure  a  contract  in  order  to  align 
contractor  incentives  with  the  government's 
overall  goals. 


1®  There  are  multiple  reviews,  boards,  and  courts  to 
which  the  firm  may  appeal.  Unless  the  government 
can  prove  outright  fraud,  firings  are  extremely  rare, 
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even  on  programs  with  huge  cost  and  schedule 
overruns  or  performance  flaws. 


•  Reorient  Incentives  Towards 
Software  Quality:  The  incentives  for 
industry  can  be  modified  simply  by 
restructuring  the  bases  for  their  payment. 
There  are  two  parts  to  the  shift  in  incentives: 
First,  cost  and  schedule  incentives  would 
have  to  be  de-emphasized.  Second,  the 
immeasurability  of  quality  requires  a  different 
path  than  a  simple  shift  to  quality  incentives. 
Emphasis  on  quality  should  occur  through  an 
indirect  link  to  something  more  measurable, 
such  as  Software  Process  Improvement 
(SPI),  an  approach  that  encourages 
contractors  to  embrace  software  development 
practices  believed  by  many  practitioners  to  be 
essential  to  developing  "good  software." 

Software  process  improvement  offers  the 
means  for  shifting  how  money  is  spent  over 
the  life  cycle  of  a  system  by  making  it 
worthwhile  for  firms  to  establish  more 
rigorous  software  development  processes  and 
to  increase  training  of  development 
personnel.  SPI  consists  of  many  activities  by 
which  firms  can  improve  the  processes, 
technologies,  and  quality  of  the  workforce 
they  employ  to  produce  software  systems.  It 
is  an  attempt  to  bring  engineering  rigor  or  the 
discipline  of  science  to  the  development  of 
software.  The  end  goal  is  higher  quality 
software  developed  under  conditions  of  more 
predictable  cost  and  schedule  performance. 
A  software  process  consists  of  those 
processes,  methods,  or  tools  that  establish 
the  managerial  and  technical  environment  in 
which  software  products  are  developed, 
including  associated  products  (project  plans, 
schedules,  design  documents,  code,  test 
cases,  user  manuals,  and  meeting  minutes  are 
some  examples)  [31-32]. 

The  Defense  Department's  Software 
Engineering  Institute  (SEI)  spawned  the  SPI 
industry  with  its  initial  version  of  the 
Capability  Maturity  Model  (CMM)H  for 


UThe  Capability  Maturity  Model  (CMM)  is  the 
product  of  decades  of  software  development  and 
management  experience  by  Watts  Humphrey,  the 
Software  Engineering  Institute,  and  many  current  and 
former  IBM  software  managers  and  practitioners 
married  to  the  quality  control  principles  of  Crosby 


software.  The  SEI  and  CMM  were  created  to 
help  the  Defense  Department  and  its  industrial 
suppliers  understand  and  control  the  plethora 
of  acquisition  problems  in  building  very  large 
software  systems.  The  CMM  is  used  as  a 
guide,  framework,  or  point  of  reference  for 
most  SPI  programs.  Currently,  the  software 
industry  spends  easily  more  than  100  million 
dollars  per  year  on  SPI  activities. 

The  CMM  is  a  normative  collection  of 
observable  software  development  and 
management  practices  that  are  believed  by 
many,  particularly  advocates  of  "software 
engineering,"  to  produce  better  software. 
The  CMM  is  not  a  validated,  operational 
model  for  companies  to  optimize  their 
software  development  processes.  The  model 
consists  of  five  basic  levels  of  process 
maturity  to  capture  the  progression  from  an 
ad  hoc  software  process  to  one  that  is  under 
statistical  control  and  a  stable  foundation  for 
continuous  process  improvement.  Each 
maturity  level  is  made  up  of  a  varying 
number  of  key  process  areas  representing 
different  facets  of  the  software  development 
process  such  as  project  planning, 
configuration  management,  and  defect 
prevention.  The  model  is  organized  to  give 
contractors  incentives  to  invest  in  greater 
levels  of  software  maturity. 

By  using  SPI  as  a  proxy  for  software 
quality,  software  quality  is  being  put  on  more 
equal  footing  with  the  more  measurable 
dimensions  of  performance.  Offering 
incentives  for  SPI  can  be  done  without  much 
political  and  cultural  change.  However, 
experience  using  SPI  as  part  of  an  incentive 
package  is  limited. 

The  first  two  solutions  possess  some  merit  in 
resolving  the  problem  of  acquiring  quality 
software  products;  however,  the  government 
and  industry's  record  regarding  procurement 
reform  over  the  last  four  decades,  in  terms  of 
changing  laws,  regulations,  and  culture  to 
make  the  processes  more  efficient,  has  been, 
at  best,  slow.  Furthermore,  each  would 
require  time  consuming  legal  changes  by 
Congress  and  cultural  changes  in  the 


[33],  Juran  [34],  and  Deming  [35].  The  initial 
version  of  the  model  is  detailed  in  [36]. 
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government's  procurement  agencies.  A  less 
than  perfect  alternative,  restructuring  a  firm's 
incentives  towards  software  quality,  may  be  a 
politically  and  legally  feasible  policy  option. 

4.0  A  Principal-Agent  Model  For 
Government  Software  Procurement 

There  are  important  facts  of  life  in  software 
procurement.  Quality  is  much  more  difficult 
to  measure  than  the  scheduling  and  budgeting 
aspects  of  performance.  Quality 
improvement  is  inherently  risky.  The 
government  tends  to  underreward  investment 
in  software  quality. 

The  software  acquisition  relationship  is 
modeled  using  a  specialized  version  of 
Holmstrom  and  Milgrom's  [12]  principal- 
agent  model.  The  specialization  consists  of 
adding  a  third  dimension  to  account  for  SPI. 
The  model  is  used  to  explore  the  conditions 
under  which  it  is  in  the  government's  interest 
to  offer  an  incentive  for  SPI. 

4.1  Tension  Between  Quality  and 
Timeliness:  In  the  procurement  of 

software  systems,  a  tension  exists  between 
the  most  measurable  dimension  of 
performance,  timeliness,  12  and  the  least 
measurable  dimension,  quality.  Government 
contracts  typically  incorporate  measures  of 
timeliness,  but  little  or  nothing  on  measures 
of  software  quality. 

Arrow  [37]  argued  theoretically  that  there  is  a 
tendency  to  underinvest  in  risky  tasks.  From 
an  empirical  perspective,  Myers'  [17] 
indicated  that  effort  on  software  quality  is  a 
risky  proposition  and  that  firms  typically 
choose  to  emphasize  timeliness, 
underinvesting  in  quality.  The  exchange  of 
quality  for  timeliness  takes  the  form  of 
departures  from  good  software  development 
practices.  Some  examples  are: 


12For  simplicity,  we  use  "timeliness"  as  a  single 
variable  denoting  all  of  the  easily  measurable 
dimensions  of  performance,  most  notably  cost  and 
schedule  parameters.  Including  two  or  more 
measurable  dimensions  would  complicate  the 
modeling  needlessly.  From  a  pragmatic  point  of 
view,  this  simplification  is  appropriate  since  cost  and 
schedule  performance  are  strongly  correlated  in 
software  procurements. 


•  prototyping  is  scratched  from  the  plan 
even  though  the  customer  has  little 
understanding  of  what  should 
constitute  the  core  set  of  initial 
requirements, 

•software  design  is  completed  and 
programming  initiated  before 
important  requirements  have 
stabilized, 

•significant  programming  is  completed 
before  interfaces  between  the 
system's  main  components  have  been 
defined,  and 

•the  testing  program  is  collapsed  into  a 
single  system  testing  phase. 

These  practices,  each  of  which  favorably 
affects  immediate  schedule  and  budget 
constraints,  will  usually  lower  the  quality  of 
the  software  produced. 

4.2  Specializing  the  Holmstrom 
and  Milgrom  Model:  Holmstrom  and 
Milgrom  (H&M)  deviate  from  traditional 
principal-agent  models  in  that  they  consider 
situations  where  the  principal  desires  the 
agent  to  perform  a  single,  multi-faceted  task 
[12].  Their  model  is  a  good  framework  for 
introducing  observables  and  conducive  to 
studying  parameters  with  varying  degrees  of 
measurability  (quality  versus  timeliness), 
including  parameters  that  cannot  be  measured 
at  all  [12].  The  linear  incentive  structure  of 
their  model  appears  to  be  a  good 
approximation  to  government  software 
contracting,  where  contractors  are  typically 
paid  for  the  accomplishment  of  each  of  the 
many  decomposed  tasks  that  comprise  the 
total  procurement  contract.  The  key 
assumptions  of  the  H&M  model  are:  1)  risk 
adverse  contractor,  2)  risk  neutral 
government  customer,  and  3)  effort  is  not 
observed  directly  by  the  government,  but  a 
signal  related  to  effort  is  observed.  Using  a 
very  general  model,  they  show  that  when  an 
contractor  performs  a  multi-faceted  task, 
linear  incentive  payments  optimally  balance 
the  trade  off  between  providing  incentives 
versus  not  putting  too  much  risk  on  the 
contractor. 
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•  Holmstrom  and  Milgrom  Model: 

Define  the  following: 

t  =  vector  containing  the 
contractor's  effort  selections  for  each  attribute 
important  to  the  government 

Y  =  is  a  vector  of  normally 
distributed  signals  the  government  observes 
for  each  of  the  contractor's  effort  selections 
for  each  attribute  (q(t)  +  e,  where  e  ~  N[0, 
Z]) 

W(Y)  =  contractor's  wages  based 
upon  signals  or  metrics,  Y ,  of  the 
contractor's  efforts 

a  =  vector  containing  the 
incentives  provided  by  the  government  for 
each  attribute 

b  =  scalar  indicating  the  sharing 
rule  allocating  the  joint  benefits  between  the 
government  and  contractor 

r  =  scalar  measure  of  the  degree 
of  the  contractor's  risk  aversion 

B(t)  =  benefits  that  accrue  solely  to 
the  government  from  the  contractor's  efforts 
t-  B(t)  is  assumed  to  be  concave  in  its 
arguments. 

C(t)  =  costs  the  contractor  incurs 
based  upon  its  effort  selections.  C(t)  is 
assumed  to  be  strictly  convex  in  its 
arguments. 

The  contractor  examines  the  compensation 
contract,  W(Y),  the  associated  costs,  C(t), 
for  expending  effort,  and  selects  an  effort 
allocation  to  maximize  his  profit: 

E[_e-w(Y)j  _  C(t)  (la) 

All  benefits,  B(t),  accrue  entirely  to  the 
principal;  thus,  she  chooses  W(Y)  (i.e.,  a 
function)  in  order  to  maximize  her  returns: 

E[B(t*)-W(Y)]  (lb) 

where  t*  is  the  solution  to  (la)  and, 
consequently,  dependent  upon  W(Y). 

From  H&M,  we  know  the  contractor's 
wages  depend  upon  the  incentives,  signals 
observed  by  the  government,  and  a  scalar 


allocating  the  total  certainty  equivalent 
between  the  two  parties: 

W(Y)  =  aTY  +  /3  (2) 

Consequently,  [la  &  lb]  have  the  same 
solution  as  that  found  by  solving  for  the  total 
certainty  equivalent  (joint  surplus): 

max  B(t)  -  C(t)  -  ^raTZcc  (3) 

a,t  c_ 

subject  to  the  following  constraint: 

t  maximizes  aT/r(t)  -  C(t)  (4) 

where  aTZa  is  the  variance  of  the  agent's 
wages  under  the  linear  incentive  structure. 

•  An  Incentive  For  Software 
Process  Improvement  (SPI):  The 

specialization  of  the  H&M  model  centers  on 
the  government's  desire  that  the  contractor 
expend  effort  on  SPI  because,  as  will  be 
shown,  SPI  has  an  overall  positive  effect  on 
software  quality.  We  now  make  some 
important  assumptions  for  modeling 
government  software  procurement:  first,  this 
problem  involves  three  attributes:  timeliness, 
quality,  and  SPI,  where  the  effort  expended 
on  each  activity  is  represented  by  tj,  t2,  and 
t3,  respectively;  second,  software  quality  is 
difficult  to  measure  and  most  organizations 
perform  little  or  no  quality  measurement  [16, 
38];  and  third,  the  government  derives  no 
direct  benefit  from  SPI.  The  associated 
incentives  for  timeliness,  quality,  and  SPI  are 
ai,  a.2,  and  ot3,  respectively. 

The  government  derives  no  direct  benefit 
from  SPI.  The  single  most  important 
assumption  in  this  model  is  that  B3,  or  the 
partial  derivative  of  the  benefit  function  with 
respect  to  t3,  equals  0.  Nevertheless,  the 
government  desires  the  contractor  to  expend 
effort  on  SPI  because  they  know  it  generally 
improves  quality  and  timeliness.  The 
government  can  only  derive  direct  benefit 
from  contractor  effort  expended  on  timeliness 
and  quality.  Thus,  we  show  that  even 
though  the  government  does  not  benefit 
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directly  from  SPI,  the  government  will  still 
find  it  desirable  to  reward  efforts  for  SPI. 

The  other  important  procurement-related 
assumption  is  the  difficulty  of  measuring 
software  quality  in  a  procurement.  It  is 
difficult  to  measure  relative  to  the  other 
dimensions  of  our  model  of  government 
software  procurement.  To  capture  this 
phenomenon  in  our  model,  we  assume  the 
variance  of  measuring  software  quality  is 
significantly  larger  than  the  variances  for 
measuring  timeliness  and  SPI. 

Using  the  above  assumptions,  we  now  solve 
(3)  subject  to  (4)  to  arrive  at  the  optimal 
incentive  for  SPI.  A  complete  derivation  of 
the  incentive  for  SPI  is  provided  in  [39]. 

5.0  Incentives  for  SPI 

We  now  explore  the  conditions  under  which 
the  government  should  pay  a  contractor  an 
incentive  for  SPI,  an  activity  for  which  the 
government  derives  no  direct  benefit.  The 
government  faces  a  dilemma  analogous  to  the 
private  market  failure  faced  by  insurers. 
Greenwald  and  Stiglitz  showed  that 
commodities  that  decrease  the  frequency  and 
size  of  insured  losses  should  be  subsidized, 
including  the  payment  of  a  subsidy  by  the 
government,  to  improve  overall  welfare  [40]. 
SPI  is  just  such  a  commodity,  helping  an 
organization  to  improve  its  processes  and 
product  quality  which  should,  in  turn, 
improve  measures  of  timeliness. 

Recent  experiences  with  SPI  provide  strong 
evidence  that  as  the  effort  expended  on  SPI 
increases,  the  marginal  costs  for  timeliness 
and  quality  decrease.  What  is  not  clear  is 
whether  timeliness  is  affected  directly  by  SPI 
or  as  a  result  of  improved  quality.  13  Dion 
[41]  has  provided  impressive  quantifiable 
evidence  of  a  near  700  percent  return  on 
investment  and  better  schedule  performance 
from  SPI  activities  by  examining  the 
counterfactual:  "What  if  the  organization  had 
not  done  SPI?"  Dion's  experiences  at 
Raytheon  along  with  the  lessons  learned  from 


^Improved  quality  may  affect  timeliness  by  causing 
the  respective  firm  to  spend  less  time  on  testing  and 
reworking  activities. 


other  SPI  efforts  provide  strong  emerging 
evidence  that  SPI  improves  software  quality 
and  measures  of  timeliness  [22,  36,  and  42- 
50], 

Additionally,  Besselman  and  Rifkin  [38] 
used  SPI  results  from  51  organizational 
evaluations  to  find  that  organizations  with 
their  operations  organized  around  a  product 
line  possess  greater  software  process 
maturity.  The  more  mature  organizations 
possess  a  greater  incentive  to  pursue  SPI. 
They  are  better  able  to  internalize  the  costs  of 
SPI  and  derive  benefits  over  the  life  cycle  of 
their  products  as  a  result  of  their  product  line 
focus.  The  evidence  is  that  increasing  effort 
on  SPI  lowers  the  marginal  costs  of  software 
quality. 

The  incentive  for  SPI  is  now  examined  under 
a  variety  of  measurability  conditions  for 
timeliness,  software  quality,  and  SPI. 

a)  How  does  the  incentive  for  SPI 
vary  with  changes  in  the  ability  to  measure 
timeliness  and  SPI?  This  is  a  difficult 
relationship  to  solve  analytically,  so  we 
simulated  the  this  relationship  using  the 
derived  optimal  incentive  for  SPI.  Figure  1 
shows  the  simulation  results. 


Figure  1:  Change  in  the  magnitude  of 
the  incentive  for  SPI  with  changes 
in  the  measurability  of  timeliness 
and  SPI. 

The  simulation  reveals  that  the  incentive  for 
SPI  is  not  dependent  upon  the  measurability 
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of  timeliness.  As  SPI  itself  becomes  more 
difficult  to  measure,  the  incentive  for  SPI 
approaches  zero. 

The  measurability  of  SPI  dictates  the 
magnitude  of  the  incentive  for  SPI.  The 
issue  of  effectiveness  centers  on  the  need  for 
a  reliable  method  for  measuring  SPI  in  an 
organization,  including  competent 
government  oversight  personnel  for 
effectively  interpreting  and  using  the  results. 
At  this  point,  there  is  not  an  extensive  and 
rigorous  body  of  evidence  telling  us  to  what 
extent  measured  SPI  predicts  quality  in 
software  outcomes.  Clearly,  pilot  SPI 
programs  should  be  designed  to  validate  the 
relationship  between  measured  software 
process  characteristics  and  software 
outcomes,  and  should  experiment  with  a 
variety  of  measurement  approaches  from 
surveys  to  the  current  practice  of  on-site 
evaluation  teams. 

b)  How  does  the  incentive  for  SPI 
vary  with  the  measurability  of  software 
quality?  In  this  case,  we  relax  our  earlier 
assumption  that  software  quality  is  difficult  to 
measure.  This  relationship  is  captured  in 
Figure  2. 


Figure  2:  Change  in  the  incentive  for 
SPI  with  changes  in  the 
measurability  of  software  quality. 

As  software  quality  becomes  more 
measurable,  the  incentive  for  SPI,  should  be 
decreased.  Again,  the  measurability  for 
timeliness  also  had  no  effect  on  this 
simulation. 


c)  How  does  the  incentive  for  SPI 
vary  with  changes  in  the  measurability  of 
software  quality  and  SPI?  Figure  3  captures 
this  relationship.  Unlike  the  finding  from 
Figure  2,  the  incentive  for  SPI  is  affected  by 
changes  in  its  measurability.  As  SPI 
becomes  more  measurable  and  software 
quality  becomes  immeasurable,  one  would 
want  to  offer  a  greater  incentive  for  SPI.  As 
expected,  when  SPI  becomes  immeasurable, 
one  would  not  offer  an  incentive,  regardless 
of  the  measurability  of  software  quality. 


Decreasing 

Measurability 

of 

SPI 
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Figure  3:  Change  in  the  incentive  for 
SPI  with  changes  in  the 
measurability  of  software  quality 
and  SPI. 

We  can  also  show  that  increasing  effort  on 
software  quality  results  from  increasing  the 
incentive  for  SPI  (see  [39]  for  a  formal 
proof).  Thus,  if  the  incentive  for  SPI 
increases,  so  does  the  investment  in  quality, 
and  hence,  so  does  the  average  quality  of  the 
software. 

The  intuition  behind  our  results  is 
straightforward.  By  providing  incentives  for 
SPI,  the  government  indirectly  encourages 
the  software  developer  to  increase  effort  on 
improving  quality.  As  our  results  have 
shown,  the  greater  the  reduction  in  marginal 
cost  of  quality  from  SPI,  the  bigger  the  effect 
of  encouraging  SPI  on  the  agent's  efforts  to 
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quality,  and  hence  the  greater  should  be  the 
government  incentive  for  SPI.  Our  results 
also  show  that  the  greater  the  precision  in 
measuring  SPI,  the  greater  should  be  the 
government's  incentives  for  SPI.  Thus,  SPI 
works  if  it  lowers  the  marginal  cost  of 
improving  quality  and  if  it  can  be  measured 
more  precisely  than  software  quality  can  be 
measured. 

6.0  Recommendations  for  Reform 

Our  recommendations  are  drawn  from  the 
findings  yielded  by  the  model.  First,  the 
government  should  consider  offering 
incentives  for  SPI  in  software  intensive 
procurements,  particularly  those 
procurements  where  contractors  are  being 
brought  on  for  short  periods  of  time  and  in  a 
time  and  materials  contract  capacity.  Second, 
the  government  should  diminish  the  emphasis 
on  cost  and  schedule  incentives  to 
accommodate  SPI.  Third,  when  a 
government  program  office  chooses  this 
course  of  action,  they  need  to  ensure  they  are 
using  reliable  evaluation  techniques  and 
trained  evaluation  teams.  Our  findings 
clearly  indicate  that  the  quality  of  the 
government's  software  management 
personnel  affects  whether  it  is  practical  to 
offer  an  incentive  for  SPI. 

The  main  attraction  of  our  recommendations 
is  that  the  current  body  of  procurement  rules 
and  regulations  can  easily  accommodate 
them.  To  include  incentives  for  SPI  in 
software  intensive  procurements  requires  no 
legal  intervention  from  the  Congress  or  the 
various  procurement  agencies  of  government. 
The  principal  change  from  SPI  is  a 
restructuring  of  the  payment  pools  in  the 
contracts  written  by  government  managers. 
One  government  acquisition  organization,  the 
U.S.  Air  Force,  has  already  begun 
experimenting  with  SPI  incentives  and 
several  other  organizations  are  considering 
them.  SPI's  political  feasibility  is  a  main 
attraction.  We  hold  no  illusions  that  far 
reaching  reforms,  two  of  which  we  discussed 
in  Section  3.0,  will  take  place  anytime  soon, 
so  we  have  focused  on  feasibility. 

7.0  Conclusions 

This  paper  modeled  the  relationship  between 
timeliness,  software  quality,  and  SPI  in 


software  procurements  to  understand  the 
conditions  under  which  the  government 
should  offer  incentives  for  SPI.  The  key 
assumptions  in  modeling  were  that  SPI  will 
enhance  the  quality  of  a  product  and  result  in 
better  cost  and  schedule  performance  all  the 
while  the  government  derives  no  direct 
benefit  from  SPI.  More  specifically,  the 
justification  for  the  government's  offering 
incentives  for  firms  to  engage  in  SPI 
activities  hinges  on  whether  increases  in 
effort  on  SPI  decreases  the  marginal  cost  of 
software  quality  and  on  whether  the 
government  will  be  able  to  measure  SPI 
effectively.  Several  citations,  largely 
consisting  of  anecdotal  findings  and  small 
sector  studies,  were  provided  indicating  that 
increases  in  effort  for  SPI  decreases  the 
marginal  cost  of  software  quality.  There  is 
not,  to  our  knowledge,  any  rigorous, 
industry-wide  empirical  evidence  regarding 
this  relationship. 

Empirical  work  is  badly  needed.  How  does 
SPI  affect  the  production  function  for 
software?  Does  SPI  help  or  hinder  the 
development  of  better  quality  software?  Does 
it  depend  on  organization  style  or  size  or  type 
of  software  being  developed? 

The  government  should  experiment  with  SPI 
on  several  new  procurements  involving  firms 
at  all  levels  of  software  process  maturity. 
The  theoretical  work  in  this  paper  sets 
priorities  for  the  empirical  work:  How  does 
an  increase  in  effort  for  SPI  affect  the 
marginal  cost  of  quality  and  timeliness? 
What  are  the  magnitudes  of  the  effects? 
There  should  be  a  systematic,  industry-wide 
study  of  the  effect  of  SPI  on  timeliness  and 
quality. 

Even  though  the  government  derives  no 
direct  benefit  from  SPI,  the  government  may 
be  justified  in  offering  SPI  incentives  to 
firms.  The  government  systematically 
underinvests  in  quality  while  acquiring 
software.  This  underinvestment  significantly 
delays  and  increases  the  life-cycle  costs  of 
software.  The  difficulties  in  measuring 
software  quality  and  incorporating  measures 
of  software  quality  into  contractual 
relationships  is  a  primary  contributor  to  the 
government's  underinvestment  in  software 
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quality.  SPI  may  effectively  serve  to 
improve  software  quality.  Thus, 
organizations  that  undertake  SPI  programs 
may  be  expected,  on  average,  to  produce 
higher  quality  software.  There  are 
measurable  characteristics  of  software 
development  organizations  that  are  widely 
believed  to  be  associated  with  producing 
better  quality  software.  These  associations 
must  be  examined  empirically. 

However  effective  SPI  is,  it  can  solve  only 
part  of  the  government’s  software  acquisition 


problems.  The  government  still  needs  to 
address  other  fundamental  flaws  in  the  way  it 
contracts  for  software  to  reduce  costs  and 
time  to  development  while  increasing  quality. 
The  most  obvious  flaws  are  one  to  five-year 
contract  increments  and  the  separation  of  the 
development  and  maintenance  phases, 
precluding  the  government  from  taking  a  life- 
cycle  perspective  in  software  acquisition. 
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Abstract 

Using  emerging  evidence  of  the  benefits  of 
software  process  improvement,  we  present  a 
theory  for  why  the  software  development 
community  is  witnessing  differential  results 
from  its  software  process  improvement 
efforts.  Our  theory  predicts  that  firms 
organized  around  product  lines  are  more 
likely  to  possess  higher  levels  of  maturity. 
Using  the  results  of  51  software  evaluations 
and  assessments,  a  statistical  test  fails  to 
reject  the  theory.  Our  findings  lead  us  to 
recommend  that  the  government  should  offer 
incentives  for  software  process  improvement 
in  software  intensive  procurements,  embrace, 
where  appropriate,  a  product-line  approach  to 
procurement,  and  refrain  from  placing 
software  quality  requirements  in  contracts. 

Introduction:  There  is  a  great  deal  of 
uncertainty  surrounding  the  actions  a  firm 
must  take  to  improve  its  software  processes 
and  realize  a  positive  return-on-investment 
from  software  process  improvement 
activities.  The  landscape  is  replete  with  firms 
that  have  learned  that  successful  software 
process  improvement  does  not  merely  consist 
of  performing  the  practices  in  the  Capability 
Maturity  Model  (CMM)  for  Software  [1]. 
Issues  of  leadership,  business  strategy, 
software  technology,  market  forces, 
relationships  and  leverage  with  the  customer, 
and  a  plethora  of  human  and  organizational 
issues  appear  to  play  a  role  in  how  successful 
a  firm  is  at  improving  its  software  processes 
and,  ultimately,  its  financial  position. 

This  paper  provides  insights  into  why  some 
firms  are  leaving  the  majority  behind  as  they 
mature  organizationally  and  reap  substantial 


benefits  from  what  they  regard  as  their 
software  process  improvement  activities. 
These  insights  deserve  careful  consideration 
as  the  government  attempts  to  craft  software 
procurement  reforms  to  take  us  into  the  next 
Century. 

Using  findings  from  government-led 
software  capability  evaluations  and  firm- 
sponsored  software  process  assessments 
collected  over  the  last  four  years,  we  find  that 
higher  maturity  organizations  are  either 
developing  commercial  products  1  or  have 
their  government  procurements  structured 
around  a  development  that  can  be 
characterized  as  a  product  line.  Embodied  in 
the  product  line  focus  is  significant  reuse  of 
software  components  and  architectures.  As 
an  example,  so  great  are  the  benefits  and 
intense  is  the  competition  in  the  international 
air  traffic  control  system  market,  that  a 
handful  of  high  maturity  firms  have  seen  their 
development  costs  drop  from  the  many 
hundreds  of  million  of  dollars  to  the  tens  of 
millions  of  dollars  per  system  in  the  span  of 
five  years. 

This  paper  examines  data  we  have  collected 
over  the  last  few  years  and  offers  a  theory  for 
why  this  focus  on  product  lines  may  be 
occurring.  At  this  point,  we  cannot 
determine  whether  software  process  maturity 
causes  an  organization  to  focus  development 
around  product  lines  or  a  business  strategy  of 
product  line  focus  motivates  stronger  efforts 
at  software  process  improvement.  We  do 
believe,  however,  that  economics  is  the 
motivator  for  both  strategies,  so  we  explored 
the  economics  of  simple  incentive  contracts 
with  extensions  to  accommodate  the  effects 
of  software  process  improvement  (SPI). 

An  analysis  of  the  economics  reveals  that 
companies  are  able  to  develop  stronger 
process  improvement  programs  because  of 
their  ability  to  amortize  their  process 
improvement  costs  over  a  larger  product  base 


^By  commercial  we  mean  non-government  sector,  not 
necessarily  the  "shrink  wrap"  software  sector.  Though 
the  findings  are  equally  applicable  to  the  shrink  wrap 
developers,  they  are  not  heavily  represented  in  our 
sample  of  firms. 
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and  derive  life  cycle  benefits.  These 
organizations  realize  these  benefits  because  of 
today's  iterative  software  development 
processes,  where  multiple  releases  are 
making  individual,  typically  government 
projects  more  similar  to  commercial  products. 

This  paper  will  open  with  a  brief  description 
of  the  problem  domain  and  why  it  is  an 
important  public  policy  issue.  We  then 
examine  the  field  survey  data,  including  a 
brief  discussion  of  the  data’s  limitations.  A 
model  of  government  procurement  is  then 
presented  using  a  simple  incentive  contract. 
The  model  is  an  extension  of  the  simple 
incentive  contract  used  by  economists 
studying  defense  procurement.  This  model  is 
augmented  by  introducing  variables  to 
account  for  how  SPI  affects  the  procurement 
equation.  We  then  discuss  each  of  the 
variables  in  light  of  current  procurement 
realities  and  our  current  knowledge  of  SPI. 
The  paper  closes  with  several  public  policy 
recommendations. 

Importance  as  a  Public  Policy  Issue: 
With  nearly  all  of  the  government  software 
contractors  and  most  commercial  contractors 
engaged  to  some  degree  in  a  SPI  program,  an 
enormous  amount  of  money  is  being  spent  by 
government  and  industry  on  SPI.  Even 
Microsoft,  the  firm  that  apparently  used  to 
disdain  process  [2],  is  now  heavily  involved 
in  its  own  form  of  process  improvement 
within  one  product  line. 2  The  specific 
motives  for  each  company  pursuing  SPI 
vary: 

•  Some  organizations  are  engaged  in  this 
activity  because  they  want  to  continue  to 
reap  the  productivity  benefits  they  have 
been  experiencing 

•  Some  are  continuing  their  genuine  efforts 
despite  achieving  mixed  results 


^Microsoft's  new-found  affinity  for  the  software 
process  was  presented  in  a  talk  by  MIT  historian  of 
technology  Michael  Cusumano  at  Carnegie  Mellon 
University  on  January  16,  1995.  His  talk  revealed 
that  within  the  last  five  years  Microsoft  had 
"discovered"  the  value  of  process  and  iterative 
software  development  as  their  products  grew  in  size,  a 
hard  lesson  countless  other  software  developers  have 
learned  over  the  last  three  decades. 


•  Some  are  finally  initiating  programs  in 
order  to  emulate  some  of  their  more 
successful  competitors 

•  Some  continue  to  watch  closely, 
experiment,  and  “hire  the  best  people” 

•  Some  lay  claim  to  "inventing"  process  in 
order  to  further  differentiate  themselves  in 
the  marketplace 

•  Some  do  not  really  believe  in  SPI,  but  put 
forth  a  modicum  of  effort  in  order  to  fill  a 
square. 

The  fact  remains,  however,  that  if  you 
examine  nothing  more  than  the  number  of 
people  attending  process  improvement 
conferences  and  workshops,  then  easily  in 
excess  of  100  million  dollars  a  year  is  spent 
on  labor  costs  alone  for  SPI.  If  you  factor  in 
the  actions  taken  to  facilitate  SPI,  then  many 
hundreds  of  millions  of  dollars  are  spent  each 
year  on  SPI,  making  it  an  extremely 
important  phenomenon  to  understand. 3 

One  motive,  filling  the  square,  has  come 
about  in  the  government  sector  because 
software  capability  evaluations^  are 
occasionally  used  when  the  government  is 
looking  for  a  developer  of  a  system  and  they 
are  considering  minimum  software  process 
maturity  levels  as  a  basis  for  awarding  the 
contract.  While  the  government  continues  to 
evolve  a  policy,  there  exists  little  theoretical 
or  empirical  understanding  other  than  that 
offered  by  Arora  et  al.  [3],  Flowe  and 
Thordahl  [4],  Herbsleb  et  al.  [5],  and  Dion  [6 


3This  is  a  rather  conservative  estimate.  The  most 
recent  Software  Engineering  Process  Group  National 
Meeting,  a  symposium  for  software  process 
improvement  practitioners,  had  more  than  1000 
people  in  attendance.  If  you  assume  the  average 
person's  salary,  benefits,  and  workplace  expenses 
amount  to  $120  thousand  dollars  (conservative),  then 
one  can  immediately  see  the  scale  of  expenditures  on 
software  process  improvement.  Once  all  labor  costs 
and  various  process  change  costs  are  factored  in,  $100 
million  is  an  extremely  conservative  estimate. 
^Software  capability  evaluations  are  procurement- 
related  audits  performed  by  the  government  to 
measure  the  software  process  maturity  of  a  contractor. 
They  are  used  in  source  selections  and  on-going 
software  intensive  procurements.  Software  process 
assessments  are  an  analogous  technique  used  by 
industry  to  either  measure  themselves  or  with  the 
help  of  an  outside  consultant. 
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and  7]  for  how  SPI  affects  the  economics  of 
a  software  product  and  of  a  firm.  These 
anecdotal  findings  are  not  easily  comparable 
and  consist  of  reasonable,  but  possibly 
subjective  assessments  of  cost  savings  and 
productivity  and  quality  improvements.  No 
rigorous  work  has  yet  been  done  relating 
software  process  maturity  levels  to 
procurement  outcomes. 

With  regard  to  the  government  sector, 
understanding  the  economics  is  important  for 
any  SPI  policy  because  the  government 
should  be  in  a  position  when  mandating 
practices  of  understanding  the  financial 
consequences  to  industry  and  government.  It 
is  the  mission  of  government,  working  with 
industry,  to  arrive  at  an  effective  policy  for 
motivating  software  process  improvement. 
Perhaps  Pope  John  XXIII  spoke  more 
eloquently  of  this  important  relationship: 

There  exists  an  intrinsic  connection  between 
the  common  good  on  the  one  hand  and  the 
structure  and  function  of  public  authority  on 
the  other.  The  moral  order,  which  needs 
public  authority  in  order  to  promote  the 
common  good  in  human  society,  requires 
also  that  the  authority  be  effective  in 
attaining  that  end. 

This  paper  offers  a  theory  delineating  the 
costs  and  benefits  of  SPI  coupled  with  a 
product  line  focus  for  both  parties  in 
government  software  procurements.  Armed 
with  this  information,  government  and 
industry  should  be  better  able  to  create  a 
policy  fair  to  both  parties. 

Field  Survey  Data:  A  software  capability 
evaluation  at  TRW  in  1993  provided  the 
impetus  to  initiate  this  study  and  collect  data. 
It  was  during  that  time  that  one  of  the  authors 
saw  first-hand  Walker  Royce's,  formerly  of 
TRW,  efforts  at  building  a  product  line 
around  one  of  the  developments  he  managed. 
The  process  maturity  differences  between  the 
"product  line"  and  "other"  business  prompted 
our  study. 

The  field  survey  data  consists  of  37 
government-led  software  capability 
evaluations  and  14  firm-sponsored  software 
process  assessments.  For  consistency,  the 


data  was  drawn  from  only  teams  that  were 
led  or  trained  by  one  of  the  authors.  None  of 
the  data  in  our  sample  are  drawn  from  self- 
assessments  by  a  firm. ^  Each  team  leader 
was  polled  for  information  about  each  firm: 
nature  of  products  being  developed,  number 
of  projects,  customer-base,  evidence  of 
product  line  focus,  and  maturity  level. 

Aside  from  each  firm's  maturity  level,  the 
critical  piece  of  information  sought  was 
whether  the  firm  had  the  ability  to  control  the 
life  cycle  of  their  products.  Did  the  firm 
engage  in  one-  to  five-year  contracts  with  the 
government  where  there  is  a  contractual 
partition  between  development  and 
maintenance  or  was  the  firm  servicing  a 
commercial  customer  where  development 
consisted  of  multiple  releases  built  upon  a 
core  architecture?  If  the  firm  had  a  product 
line  focus,  then  the  firm  controlled  the  life 
cycle  of  their  products;  i.e.,  they  had  the 
ability  to  derive  life  cycle  benefits  from 
greater  attention  to  quality  during 
development.  If  the  firm  developed 
commercial  products,  then  they  either 
controlled  or  had  the  ability  to  control  their 
life  cycle.  If  the  firm  relied  solely  on 
government  contracts,  possessed  little  or  no 
evidence  of  reuse,  possessed  a  wide  variation 
of  project  types,  or  lacked  a  product  line 
focus,  then  we  determined  that  they  did  not 
have  the  ability  to  derive  benefit  over  the  life 
cycle  of  their  products.  Due  to  the  expense 
and  time  required  to  collect  this  kind  of  data, 
we  relied  on  the  judgment  of  each  team  leader 
in  making  the  determination  of  whether  a  firm 
had  control  over  the  life  cycle  of  their 
products. 

The  data  is  summarized  in  a  contingency  table 
found  in  Table  1,  where  it  is  sorted  by 
maturity  levels  and  whether  an  organization 
controlled  the  life  cycle  of  their  products.  A 
"+"  attached  to  the  maturity  levels  indicates 
the  organization  was  strong  at  that  level.  No 
maturity  level  scores  extend  beyond  3+ 
because  the  teams  did  not  spend  the  time  to 
reliably  examine  practices  beyond  level  3. 


5 This  an  important  point,  for  a  problem  pervasive  in 
the  SPI  community  is  the  maturity  level  differences 
between  firm-led  assessments  and  government-led 
evaluations.  See  [8]  for  more  discussion. 
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Much  of  Table  1  consists  of  process  maturity 
information  on  a  sample  of  companies 
engaged  in  the  development  of  software 
systems  for  the  government  and  industry 
(some  drawn  from  [8]).  The  sample  is 
biased  towards  firms  developing  command 
and  control  systems  for  the  government. 
Some  of  the  companies  developing 
government  systems  also  develop  commercial 
products.  A  few  organizations  are  reflected 
more  than  once  to  account  for  different  teams 
evaluating  or  assessing  the  same  site,  usually 
with  different  project  selection  and  at  a 
different  point  in  time.  There  are  some 
limitations  to  the  data  beyond  what  has 
already  been  mentioned: 

•  The  data  points  (organizations)  are  not  a 
random  sample.  Each  organization  was, 
for  the  most  part,  an  early  adopter  of  SPI. 

•  Some  of  the  assessment  data  points 
represent  projects. 

•  Multiple  organization  and  project  sizes  are 
represented. 

•  The  data  points  were  collected  over  a  three- 
year  period. 


Maturity 

Levels  vs.  Control  of  Life  Cycle 

Maturity  Levels 

Control  of  Life  Cycle 

Yes 

No 

1 

1 

16 

1  + 

5 

4 

2 

10 

2 

2+ 

2 

0 

3 

7 

0 

3+ 

4 

0 

29 

22 

Table  1:  Field  Survey  Data  From  51 
Evaluations  and  Assessments 

Simple  Incentive  Contracts:  In  this 
paper,  the  government  engages  in  a  contract 
with  a  firm  to  develop  a  software  system. 
This  contract  is  really  a  risk  sharing 
relationship  important  to  both  parties:  the 
government,  in  its  search  for  the  perfect 
incentive  contract  motivating  a  contractor  to 
meet  its  schedule  while  making  a  fair  profit, 
appears  to  oscillate  between  fixed-price 
contracts  (contractor  bears  all  risk)  and  cost- 
plus  contracts  (government  bears  all  risk).6 


^This  oscillation  appears  to  have  been  going  on  since 
the  1950s  when  Peck  and  Scherer  [1962]  first 


Since  Peck  and  Scherer  [10]  performed  their 
pioneering  economic  analysis  on  defense 
procurement,  many  economists  have  revisited 
this  problem  every  few  years.  Scherer  [11] 
introduced  the  basic  model  of  a  firm’s  profit 
function  and  the  government’s  associated 
cost  function: 

Ilf=nt  +  a(Ct-Ca)  (1) 

where  Flf  is  the  firm’s  profit,  Fit  is  the  firm’s 
target  or  negotiated  profit,  “a”  is  a  sharing 
parameter  between  the  government  and  a 
firm,  Ct  is  the  target  or  negotiated  cost  of  the 
system,  and  Ca  is  the  actual  cost  of  the 
acquisition.  This  simple  incentive  contract 
represents  some  of  the  contract  types 
encountered  in  government  procurements. 
When  a=l,  the  contract  type  is  firm-fixed 
price.  When  a=0,  the  contract  type  is  cost- 
plus  fixed  fee.  When  “a”  is  allowed  to  vary 
between  0  and  1,  it  is  viewed  as  a  simple 
incentive  contract.  Much  of  the  research  over 
the  years  has  been  on  determining  what  is  the 
optimal  value  “a”  for  the  government  and 
each  firm  [See,  for  example,  11-15]. 
Unfortunately,  the  findings  have  been 
inconclusive  and  sometimes  contradictory. 

Extending  The  Simple  Model  To 
Account  For  SPI:  We  now  explore  how, 
if  at  all,  does  SPI  affect  this  equation.  Dion 
[1993:  28]  has  provided  impressive 

quantifiable  evidence  of  near  800  percent 
return  on  investment  and  better  schedule 
performance  from  SPI  activities  by 
examining  the  counterfactual:  what  if  we  had 
not  done  SPI?  Dion's  experiences  at 
Raytheon,  along  with  our  experiences  and  the 
lessons  learned  from  other  SPI  efforts,  offer 
a  number  of  financial  opportunities  [6, 7,  and 


documented  it.  Roughly,  the  excess  profits  in  the 
defense  industry  during  the  50’ s  and  70's  from  cost- 
plus  contracts  were  followed  by  periods  of  fixed-price 
contracts  in  the  60’ s  and  80's,  respectively.  This 
practice  appears  to  have  saddled  many  defense 
contractors  with  heavy  debt  and  unprofitable  programs 
in  the  80's  [9].  Despite  the  wealth  of  research  in  the 
area  of  efficient  incentive  contracts,  the  government 
has  oscillated  back  to  cost-plus  contracts  for  most 
developments  in  the  90's. 
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16-19]  from  which  one  may  extend  equation 

(1): 

•  Firm’s  substantiated  software 
process  improvements  bring 
prestige  and  new  business  to  the 
firm,  P; 

•  Firm’s  process  improvement 
activities  enhance  current 
operations,  improving  immediate 
cost  and  schedule  performance,  R; 

•  Firm’s  process  improvement 
activities  cause  dissemination  of 
lessons  learned  across  the 
organization  (spillover),  resulting  in 
higher  profits,  S;  and 

•  Firm’s  process  improvement 
activities  result  in  life  cycle  cost 
savings  from  defect  reduction  and  a 
better  system  architecture,  L. 

With  these  new  variables;  prestige,  P; 
immediate  cost  savings,  R;  spillover,  S;  and 
life  cycle  savings,  L;  the  simple  incentive 
contract  model  becomes  as  follows: 

Ilf  =  lit  +  a(Ct  -  (Ca  -  R))  +  P  +  S  (2) 

This  model  provides  a  simple  description  for 
discussing  and  analyzing  the  problem 
domain.  Conspicuously  missing  from  the 
new  simple  incentive  contract  is  the  life  cycle 
savings  variable,  L.  This  purposeful 
omission  is  discussed  subsequent  to  the 
included  variables.  We  concede  that  the  new 
variables  are  interdependent  to  some  degree. 
Also,  this  is  probably  not  an  exhaustive 
accounting  of  all  variables.  One  could 
certainly  extend  or  refine  the  model  to 
account  for  behavioral  aspects  of  SPI;  e.g., 
movement  of  people  across  projects  or 
organizational  boundaries,  or  the  notion  that 
SPI  creates  value  in  people.  These  we  have 
left  for  future  research. 

Each  of  the  new  variables  added  to  the 
equation  as  a  result  of  SPI  appear  compelling 
within  the  confines  of  a  specific  acquisition, 
but  can  we  generalize  their  sign  or  magnitude 
across  all  procurements?  We  now  look  at 
each  variable  in  isolation. 


•  Prestige:  The  first  variable,  P, 
prestige  or  “new  business,”  appears  at  first 
glance  to  be  a  tangible  consideration.  There 
have  been  several  high  profile  procurements 
that  have  made  maturity  levels  a  major 
determinant  and  some  government 
procurement  organizations  have  even  called 
for  minimum  maturity  levels  in  source 
selections.  However,  there  really  has  not 
been  a  systematic  effort  by  the  government  to 
base  awards  on  maturity  levels  and/or  past 
performance.^  Procurement  officials  appear 
reluctant  to  use  measures  that  fall  under  the 
rubric  of  past  performance  [3].  Even  if  we 
are  overnight  to  make  past  performance  a 
major  consideration,  we  have  the  messy 
problem  of  determining  who  is  responsible 
for  procurement  outcomes.  How  do  we 
separate  firm  behavior  and  the  government's 
dysfunctional  oversight  practices  [20]? 

Evaluating  SPI  in  the  context  of  a  source 
selection  is  an  example  of  past  performance 
being  used.  Until  there  is  some  evidence  the 
government  is  consistently  using  past 
performance  in  source  selections  and  a  few  of 
the  more  mature  organizations  can  point  to  a 
disproportionate  number  of  source  selection 
victories,  we  have  to  assume  that  prestige  is 
zero  in  nearly  every  case. 8 

Complicating  the  introduction  of  the  prestige 
variable  is  the  use  of  software  process 
maturity  levels  by  many  firms  as  a  marketing 


^ The  problem  may  be  even  more  serious  than  meets 
the  eye.  As  recently  as  1994,  one  experienced 
government  procurement  agency  awarded  an  estimated 
$20  million  dollar  software  contract  (Ada 
development  with  a  potential  $100+  million 
procurement  tail)  to  a  firm  with  no  experience,  no 
staff  hired  at  contract  award,  and  with  a  program 
manager  that  did  not  know  what  configuration 
management  meant  after  announcing  a  12-month 
schedule  slip  a  mere  seven  months  after  contract 
award. 

^ Some  exceptions  to  this  may  be  emerging:  some 
divisions  of  Raytheon,  Hughes,  and  possibly  a  couple 
others.  They  have  taken  a  very  public  stance  on  their 
maturity(many  publications),  and  judging  from  some 
of  their  contract  awards,  they  appear  to  be  able  to 
back  it  up.  Also,  their  visible  position  undoubtedly 
has  some  effect  on  the  very  human-dependent  source 
selection  process. 
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tool.  Many  firms  are  publicly  reporting 
maturity  level  3  and  4  status  despite 
government-led  or  joint  evaluations  (whose 
results  are  not  published)  to  the  contrary.  As 
recently  as  within  the  last  quarter,  one  of  the 
largest  Software  Process  Improvement 
Network  (SPIN)  groups  hosted  a  speaker 
from  "A  Level-Three  Company"  that  had  two 
evaluations  performed  within  the  last  six 
months  that  found  it  to  be  level  1,  one  of 
which  was  a  joint  govemment/firm  team. 

The  next  new  variable,  R,  immediate  cost 
savings,  is  the  funding  saved  on  the 
respective  program  during  the  development 
as  a  result  of  SPI.  It  is  subject  to  the  sharing 
variable,  “a,”  because  cost  savings  within  an 
acquisition  are  usually  shared  with  the 
government.  This  variable  is  often  believed 
to  be  the  only  result  of  SPI.  All  indications 
are  that  it  quickly  becomes  positive  after  the 
most  rudimentary  process  improvements. 

The  last  variable,  S,  spillover,  is  positive  and 
a  fundamental  part  of  any  improvement 
effort.  One  of  the  tenets  of  SPI  is 
benchmarking  and  piloting  in  order  to  learn 
more  and  perfect  a  particular  process 
innovation.  The  “spillover”  variable  enters 
anytime  a  an  organization  begins  to  act  on  a 
successful  pilot;  i.e.,  disseminate  the 
respective  practice  throughout  its  other 
relevant  projects.  Both  R  and  S  need  to  be 
tracked  to  get  a  better  indication  for  SPI's  real 
contribution.  However,  all  of  the  variables 
discussed  so  far  pale  in  importance  to  the 
next  variable,  life  cycle  cost  savings. 

The  life  cycle  cost  savings  variable,  L,  is  the 
“action”  or  heart  of  our  thesis:  those 
organizations  building  many  unrelated 
systems  for  customers  on  support  or  time  and 
materials  contracts  normally  do  not  reap  the 
life  cycle  benefits  of  SPI.  Those 
organizations  are  typically  in  a  state  of  change 
as  their  contracts  are  frequently  modified  and 
their  personnel  shift  around  (including 
sometimes  leaving  the  company)  to 
accommodate  the  changes.  Only  those 
contractors  building  commercial  products,  or 
government  contractors  that  have  organized 
their  developments  around  a  core  product 
line,  derive  the  life  cycle  benefits. 
Consequently,  only  they  can  add  L  to  their 


equation.  The  less  fortunate  contractors  may 
put  forth  the  effort,  but  the  customer 
(government)  will  ultimately  derive  the 
benefits. 

We  represent  the  relationship  between 
benefits  from  SPI  and  SPI  effort  in  Figure  1. 
Two  benefit  functions  are  used  to  convey  the 
differences  in  returns  from  SPI  based  on 
whether  a  firm  has  instituted  a  product  line 
focus  in  their  organization. 

Arbitrary  functions  are  chosen  for  how 
benefits  of  SPI  change  as  SPI  effort 
increases.  The  two  benefit  functions  are 
concave  overall,  but,  since  it  is  assumed  that 
some  effort  must  be  expended  before  benefits 
begin  to  accrue  to  SPI,  the  benefit  functions 
are  initially  convex.  As  benefits  begin  to 
accrue,  the  benefit  functions  take  on  a 
concave  appearance  in  order  to  capture  the 
diminishing  returns  that  an  organization 
eventually  realizes.  In  other  words,  a  point  is 
reached  where  improving  a  process  requires 
increasingly  greater  effort  to  squeeze  out 
further  benefits.  Benefit  function  “A” 
represents  the  potential  benefits  a 
development  organization  may  reap  through 
successful  SPI:  better  cost  and  schedule 
performance  through  a  higher  quality  process 
and  product  for  the  respective  project  during 
development.  Benefit  function  “B”  is  nearly 
a  vertical  shift  of  “A”  to  capture  the  additional 
life  cycle  benefits  only  some  organizations 
realize,  those  that  have  organized  around 
product  lines. 
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Benefit  Function  B 


areas  where  tailoring  is  required  to  meet  the 
unique  requirements. 


BENEFITS 

OF 

SPI 


SPI 

EFFORT 


Third,  the  ability  to  derive  life  cycle  savings 
causes  a  firm  to  make  completely  different 
effort  selections  when  it  comes  to  SPI.  The 
effort  selections  will  typically  be  greater  for  a 
firm  that  has  organized  around  product  lines. 
Consequently,  the  firm's  returns  from 
prestige,  P,  internal  savings,  R,  and 
spillover,  S,  should  be  substantially  greater 
than  a  firm  with  no  concern  for  life  cycle 
savings. 

The  economic  incentives  are  such  that  the 
potential  returns  from  SPI  are  much  greater 
for  commercial  or  “product  line”  companies.9 
Given  these  conditions,  we  must  hypothesize 
the  following: 


Figure  1:  Benefits  Versus  SPI  Effort 

First,  these  product-line  oriented 
organizations  are  able  to  derive  benefits 
over  the  life  cycle  of  their  product  lines 
by  paying  stricter  attention  to  quality 
during  development.  Naturally  it  costs 
more  in  terms  of  effort  to  take  a  process 
improvement  program  to  this  level,  but  it 
is  not  nearly  as  costly  as  the  benefits  one 
derives  from  such  efforts.  Given  the 
abundance  of  data  detailing  the  benefits  of 
removing  errors  during  development 
versus  their  removal  cost  during 
maintenance,  this  increase  in  benefits  is 
also  nearly  exponential  [Weller,  1993; 
Kajiharaet  al,  1993;  Dion,  1993;  Russell, 
1991;  Mays  et  al,  1990;  and  Boehm, 
1987], 

Second,  as  part  of  their  emphasis  on  product 
lines,  these  firms  are  deriving  significant 
benefit  from  reusing  software  and 
architectures.  The  aforementioned  high 
maturity  firms  building  air  traffic  control 
systems  are  no  longer  building  systems  based 
entirely  on  custom  software.  They  are 
building  new  systems  based  upon  an 
evolving  system  baseline.  Over  time,  as 
more  and  more  customers  take  delivery  of  the 
latest  system  version,  they  are  receiving  a 
higher  quality  core  system  that  employs 
custom  software  only  in  those  functional 


Those  organizations  developing 
commercial  software  or  a 
supplier  who  has  taken  a 
“product  line”  approach  to  their 
developments  are  more  likely 
to  possess  higher  process 
maturity  levels  given  they  have 
initiated  programs  of 
improvement. 

To  test  our  hypothesis,  we  performed  a  chi- 
square  test  of  the  contingency  Table  1.  We 
should  be  able  to  reject  our  hypothesis  if 
there  are  no  significant  differences  in  maturity 
levels  between  the  two  types  of  firms 
identified  in  Table  1.  The  test  revealed 
significant  differences  in  maturity  levels  for 
those  firms  that  had  the  ability  to  derive 
benefit  from  SPI  over  the  life  cycle  of  their 
products;  i.e.,  firms  organized  around 
product  lines  were  found  to  possess  higher 


9It  has  been  alleged  elsewhere  in  the  community  that 
commercial  companies  are  lagging  behind  the  defense 
contractors  in  SPI.  We  do  not  address  the  veracity  of 
this  allegation,  as  it  is  not  critical  for  our 
conclusions.  Nevertheless,  defense  contractors  have 
had  significantly  more  opportunities  for  exposure  to 
SPI  and  certainly  “encouragement”  from  the 
government.  Technological  change  takes  time.  We 
merely  argue  that  commercial  companies  and  those 
government  contractors  with  their  developments 
organized  around  product  lines  are  ultimately  the 
greatest  beneficiaries  of  SPI. 
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levels  of  software  process  maturity  (%2  = 
38.9,  p  <  .0001).  This  finding  lends 
significant  support  to  our  hypothesis. 

Discussion:  Simply  put,  the  government 
desires  that  their  contractors  put  forth  the 
effort  in  SPI  needed  to  maximize  the 
government's  returns  from  SPI;  i.e.,  the 
government  needs  all  contractors  to  be 
concerned  about  life  cycle  cost  savings,  not 
just  the  contractors  developing  systems 
where  a  product  line  focus  is  a  natural 
strategic  decision  for  a  firm  to  make. 
Unfortunately,  for  a  variety  of  reasons,  not 
all  firms  participating  in  government 
procurement  may  derive  benefits  over  the  life 
cycle  of  a  contract.  The  short-term  duration 
for  many  software  contracts,  contractual 
separation  of  development  and  maintenance, 
use  of  time  and  materials  contracts,  and  the 
inability  of  the  government  to  effectively  use 
past  performance  [21],  even  when  firms  have 
done  a  great  job,  provides  perverse  incentives 
for  many  of  these  firms.  They  have  a 
difficult  time  justifying  effort  on  SPI,  for  the 
returns  will  accrue  entirely  to  their  customer, 
but  the  firm  will  have  to  absorb  the  cost 
internally. 

Figure  2  repeats  Figure  1  but  adds  some 
additional  lines  to  illustrate  the  change  in 
benefits  from  a  change  in  effort  and  focus  on 
product  lines  by  an  hypothetical  firm.  The 
figure  identifies  two  new  variables:  dB  and 
dE,  change  in  benefits  from  and  change  in 
effort  expended  on  SPI,  respectively. 


Figure  2:  Change  in  Benefits  Versus 

SPI  Effort 

Similarly,  we  can  represent  the  change  in 
costs  to  the  firm  as  a  result  of  increased  effort 
on  SPI.  A  firm  must  invest  in  order  to  train 
its  workforce,  study  and  change  its 
processes,  and  produce  higher  quality 
software.  The  relationship  between  a  change 
in  SPI  effort  and  increased  cost  is  captured  in 
Figure  3.  Another  new  variable,  dC,  is  used 
in  Figure  3  to  represent  the  change  in  cost 
associated  with  the  same  change  in  effort 
depicted  in  Figure  2.  The  cost  function  is 
convex.  Whether  it  is  linear  or  nonlinear 
does  not  affect  the  analysis. 

Based  upon  emerging  SPI  evidence  [3-7  and 
16-18]  and  the  existing  literature  on  the  life 
cycle  benefits  of  eliminating  defects  as  early 
as  possible  [21-25],  we  know  the  benefits 
from  SPI  are  significantly  greater  than  the 
associated  costs;  i.e.,  if  an  organization 
augments  its  SPI  efforts  toward  benefit 
function  B  and  stops  its  SPI  efforts  when 
returns  begin  to  diminish  (when  benefit 
function  B  goes  flat),  then  dB  »  dC.  From 
the  government  customer's  perspective,  as 
their  contractor  expends  dC  more  effort  on 
SPI,  the  government  will  derive  dB  dollars  in 
benefits  through  life  cycle  cost  saving.  If  the 
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contracting  firm  is  organized  around  a 
product  line,  it  will  also  derive  potentially 
greater  benefits  as  it  has  now  either 
established  a  high  quality  baseline  from 
which  to  build  future  systems  for  future 
customers  or  has  improved  upon  an  existing 
system  and  delivered  it  to  its  government 
customer  at  a  far  reduced  cost  from  that 
which  any  other  firm  could  have  delivered  at 
if  it  had  built  the  system  from  scratch.  The 
benefits  to  both  parties  in  the  contract  far 
exceed  the  costs.  The  problem  for  the 
government  is  how  to  motivate  all  contractors 
to  perform  in  a  manner  consistent  with 
benefit  function  B,  including  those  that 
presently  engage  in  contracts  where  they  are 
not  capable  of  deriving  life  cycle  benefits 
from  SPI. 


Figure  3:  Change  in  Costs  Versus 
SPI  Effort 

Public  Policy  Recommendations:  In 

translating  our  findings  into 
recommendations,  we  constrained  ourselves 
by  feasibility.  That  is,  we  sought  reforms 
that  could  be  feasibly  carried  out  with  no 
congressional  intervention.  We  want  to  give 
the  entrepreneurs  toiling  in  the  program 
offices  some  initiatives  they  could  easily 
carry  out  with  minimal  effort  to  achieve 
maximum  gain. 


There  are  two  parts  to  the  remedy  that  will 
improve  the  current  situation  for  the 
government:  motivate  SPI  and  eliminate 
dysfunctional  incentives.  These  two  parts 
translate  into  at  least  four  actions  government 
and  industry  should  embrace  to  maximize 
benefits  to  all  parties  from  SPI: 

•  Where  appropriate  provide  incentives  for 
SPI  in  government  procurements 

•  Decrease  emphasis  on  cost  and  schedule 
incentives 

•  Where  appropriate,  the  government  should 
embrace  a  product-line  approach  to 
procurement 

•  Do  not  initiate  incentives  for  defect  metric 
targets 

•  Incentives  For  SPI:  The 

government  wants  the  contractor  to  engage  in 
SPI  in  a  manner  consistent  with  the  benefits 
produced  in  benefit  function  B.  Not  all 
contractors  have  the  economic  incentives  to 
perform  at  that  level.  Even  when  contractors 
have  their  developments  oriented  around  a 
product  line,  there  is  no  guarantee  that  they 
will  perform  at  a  level  to  maximize  the 
government's  life  cycle  benefits  from  a  high 
quality  development  process  on  a  particular 
acquisition.  Despite  a  contractor’s  best 
intentions,  there  will  be  circumstances  where 
software  process  performance  abates  or 
regresses.  Consequently,  as  part  of  the 
remedy  for  this  situation,  the  government 
should  provide  an  economic  incentive  to  the 
affected  contractors  equal  to  an  estimate  of 
dC.  Even  when  paying  this  additional  sum, 
assuming  the  contractor  already  has  a  sound 
SPI  program,  this  is  a  win-win  situation  for 
the  government  and  contractor:  government 
benefits  from  greater  life  cycle  savings  and 
contractor  enjoys  more  predictable  product 
development  cycle  and  spillover  benefits. 

•  Decrease  Emphasis  on  Cost  and 
Schedule  Incentives:  Another  part  of  the 
remedy  addresses  the  other  incentives 
typically  found  in  government  procurements: 
cost  and  schedule  or  "timeliness"  incentives. 
There  is  a  tension  in  all  software 
developments  between  timeliness  and  quality. 
This  tension  arises  because  the  government  is 
quite  good  at  requiring  and  then  reviewing 
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timeliness  metrics  but  it  is  a  rare 
occurrence  when  the  government  places 
software  quality  measures  on  contract  in  the 
form  of  an  incentive.  Most  contractors  are 
very  much  aware  of  this  tension.  To  the 
detriment  of  quality,  they  usually  pay  more 
attention  to  issues  of  timeliness. 
Consequently,  we  recommend  decreased 
emphasis  on  timeliness  incentives,  including 
the  government  requiring  timeliness  metrics, 
in  software  intensive  procurements.  Instead, 
government  and  industry  should  work 
towards  a  policy  where  a  minimum  maturity 
level  is  one  of  the  bases  for  contract  award 
and  economic  incentives  for  SPI  are  part  of 
the  contract.  In  other  words,  the  government 
should  seek  to  do  business  with  contractors 
that  collect  and  use  their  own  internal 
timeliness  metrics  as  a  normal  business 
practice.  The  government  will  then  get  the 
benefits  derived  from  effective  use  of  metrics 
without  the  dysfunctional  baggage  brought 
on  by  the  side-effects  of  contractually 
requiring  certain  contractor  behaviors. 

•  Government  Product-Line 
Procurement:  The  oblique  message  of  our 
product-line  finding  is  that  the  approach  also 
makes  sense  for  the  government.  In  various 
application  domains,  the  government 
repeatedly  buys  multiple  versions  of  very 
similar  systems  comprised  of  custom  made 
software  from  many  different  firms.  Not 
only  may  this  be  fiscally  irresponsible,  but  it 
makes  no  sense  since  in  many  domains  much 
of  the  software  functionality  is  very  similar 
across  systems.  This  is  particularly  true  for 
such  domains  as  payroll,  management 
information,  communication,  command  and 
control,  and  simulator  systems. 

The  U.S.  Air  Force  has  been  trying  for 
nearly  three  years  to  institute  a  product  line 
focus  for  the  development  of  command 
centers  and  their  ancillary  functions. 
Developed  by  the  Air  Force's  Robert  Kent, 
the  Portable,  Reusable,  Integrable  Software 
Modules  (PRISM)  program  uses  two  high 


1  Opart  of  the  reason  for  this  predilection  for 
timeliness  measures  is  because  they  are  easy  to 
understand,  easy  to  gather,  and  something  managers 
in  both  government  and  industry  have  been  trained  to 
use. 


software  maturity  contractors,  Raytheon  and 
Hughes,  already  on  contract  as  the  lead 
developers  to  satisfy  all  new  Air  Force 
requirements  related  to  command  centers,  but 
they  will  use  existing,  certified  software 
components  from  already  developed  systems 
to  the  greatest  degree  possible.  1 1  They  have 
developed  a  fully  functional  command  center 
to  use  as  a  "store"  for  operational  users.  The 
"store"  utilizes  multiple  display  and  message 
passing  technologies  from  existing  programs 
along  with  pertinent  pieces  of  commercial 
software  and  hardware.  The  operational 
users  merely  pick  the  desired  pieces  of 
functionality;  they  no  longer  have  to  help 
write  a  requirements  specification,  put  up 
significant  funding,  and  wait  years  for 
custom  software  to  be  written  that  often  does 
not  meet  their  needs. 

The  benefits  are  two-fold:  First,  it  gets  the 
government  out  of  the  business  of  developing 
custom  software  that  already  exists  to 
perform  most  of  the  desired  functionality. 
Second,  the  government  already  has  two  high 
software  maturity  firms  on  contract,  so  the 
user  needs  can  be  met  much  quicker  without 
the  government  needing  to  go  through  the 
time  consuming  and  often  dysfunctional 
source  selection  process.  The  drawbacks,  in 
terms  of  cultural  change,  are  significant, 
however.  Much  of  the  procurement 
bureaucracy  has  not  been  receptive  to  this 
program.  Reusing  software  and  organizing 
around  product  lines  requires  fewer  people 
and  less  funding,  something  many  in  the 
bureaucracy  do  not  want  to  see. 

Even  though  this  is  a  pilot  test,  it  appears  to 
be  very  close  to  bearing  fruit.  The  Air 
Force's  current  requirement  to  upgrade  the 
military's  U.S. -based  air  traffic  centers 
appears  to  be  doable  using  PRISM.  The 
traditional  custom  software  and  system 


1 1  Sonic  of  he  ideas  behind  PRISM  trace  back  to  the 
Software  Engineering  Institute's  participation  in  Air 
Force's  Granite  Sentry  program  (a  command  center 
system).  The  SEI's  Michael  Rissman  led  a  team  of 
designers  that  discovered  that  much  of  the  planned 
programming  to  accomplish  various  pieces  of 
functionality  went  through  the  same  five  or  six  basic 
processing  steps. 
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approach  managed  out  of  a  government 
program  office  (SPO)  has  been  priced  in 
excess  of  $300  million.  The  PRISM 
approach  using  existing  certified  components 
currently  in  their  repository  and  commercial 
hardware  is  priced  at  under  $30  million.  If 
this  pilot  proves  successful,  12  it  should 
provide  significant  impetus  for  the  rest  of  the 
government  to  embrace  product-line 
acquisition. 

•  Do  Not  Initiate  Incentives  For 
Defect  Metric  Targets:  The  government 
should  endorse  the  use  of  quality  metrics  by 
industry,  assuming  the  respective  contractor 
has  reached  a  level  of  process  maturity  where 
the  metrics  can  be  effectively  used.  The 
government,  however,  should  not  engage  in 
placing  specific  product  quality  goals  or 
targets  on  contract  or  in  the  form  of  an 
incentive.  To  place  quality  targets  or  metric 
measure  levels  on  contract  or  in  the  form  of 
an  incentive  causes  the  following  problems: 

•  It  places  the  government  in  the  position  of 
telling  industry  how  to  administer  its 
program, 

•  It  assumes  something  about  the  respective 
contractor  or  government  program  office 
that  most  often  is  not  true — mature 
enough  to  use  the  metrics  effectively,  and 

•  It  introduces  a  host  of  dysfunctional 
incentives  [26]  to  the  contractor 
organization  to  meet  the  incentive  targets 
rather  than  the  goal  the  incentive  was 
seeking  (high  quality  products). 

Instead,  the  government  should  focus  on 
awarding  contracts  to  organizations 
possessing  high  software  process  maturity 
levels  and  ensure  the  process  maturity  is 
augmented  or  maintained  over  the  life  of  the 
respective  contract.  Higher  maturity 
organizations  will  measure  and  seek  to 
improve  quality  as  a  normal  business 
practice.  We  need  to  identify  those  firms  and 
channel  a  greater  percentage  of  our 
diminishing  funding  to  them. 


^Success  in  this  case  requires  a  broader  meaning. 
Even  if  this  pilot  exceeds  its  budget  by  100  percent, 
like  most  software  acquisitions,  it  is  still  at  least  five 
times  cheaper  than  a  traditional  SPO's  current 
estimate. 


The  four  components  of  the  remedy  are 
complementary;  i.e.,  they  should  be 
implemented  together  for  the  government  and 
industry  to  derive  the  maximum  benefit  from 
SPI. 

Future  Prospects  and  Conclusions: 

This  paper  represents  the  first  step  in 
understanding  the  economics  of  SPI.  Still, 
more  work  needs  to  be  done  to  better 
understand  how  SPI  affects  the  economics  of 
procurement.  We  have  presented  a  theory  of 
why  differential  SPI  results  are  being 
witnessed  in  the  government  contractor 
sector.  More  work  needs  to  be  done  before 
we  can  generalize  these  findings  across  all  of 
industry,  but  we  suspect  they  are  relevant  to 
organizations  that  develop  significant  pieces 
of  customized  software  for  internal  use. 
Further  work  is  also  needed  to  understand  the 
causality  of  how  SPI  relates  to  product  lines: 
Is  software  process  maturity  more  attractive 
to  product  line  builders  or  does  software 
process  maturity  cause  an  organization  to 
focus  development  around  a  product  line? 
Naturally,  we  suspect  the  latter.  Our  analysis 
also  needs  to  continue  in  order  to  develop 
estimates  by  acquisition  size  for  each  of  the 
variables  discussed. 

We  find  the  data  to  be  supportive  of  our 
theory  that  organizations  that  control  the  life 
cycle  of  their  products  possess  greater 
incentives  to  engage  in  sound  SPI  programs. 
The  government  can  reap  greater  benefits 
from  SPI  by  motivating  its  contractors  to 
continue  their  efforts  through  effective 
incentives.  The  incentive  will  have  greater 
leverage  within  companies  that  have 
organized  their  development  activities  around 
a  product  line.  This  motivation  is 
accomplished  through  economic  incentives 
for  SPI,  not  contractually-mandated  SPI 
programs  or  target  defect  levels. 

Similarly,  there  is  a  product-line  lesson  there 
for  the  government.  It,  too,  should  organize 
its  procurements  around  product  lines  using 
the  lessons  learned  from  PRISM.  This 
would  allow  the  government  to  develop  and 
deliver  high  quality  products  quicker  and 
cheaper  using  high  software  maturity  firms 
while  overcoming  the  government's 
dysfunctional  source  selection  process  and  its 
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associated  inability  to  use  past  performance 


effectively. 
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A  Model  Program  Approach 
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ABSTRACT 

The  following  is  an  outline  for  how  to 
implement  the  DoD  Cost/Schedule  Control 
Systems  Criteria  (C/SCSC)  and  related 
reports  in  a  manner  that  maximizes 
management  value  for  both  the  government 
Program  Management  Office  (PMO)  and  the 
contractor.  This  paper  is  for  discussion 
purposes  only. 

INTRODUCTION 

While  it  is  believed  that  little,  if  anything,  in 
current  policy  precludes  implementation  of 
any  of  the  following  proposals,  this  paper 
proposes  a  radical  change  in  how  we  do 
business  and  probably  cannot  be 
implemented  piecemeal.  It  requires  a 
complete  government  and  contractor 
management  commitment  to  making  earned 
value  and  fully  integrated  management 
systems,  essential  elements  of  how  we 
manage  large,  risky  contracts. 

1.  Objective:  Change  the  emphasis  from  the 
government  to  the  contractor.  C/SCSC 
compliant  systems  should  represent  how  the 
contractor  manages.  The  mere  act  of 
awarding  a  contract  should  not  trigger  a 
government  review.  Reviews  should  be 
conducted  only  for  cause. 

•  Replace  standard  C/SCSC  contract 
clause  with  a  clause  requiring  executive 
level  company  certification  that  the 
contractor's  management  systems  and 


procedures  are  fully  integrated,  reflect 
how  the  contractor  manages,  and  meet 
the  intent  of  the  C/SCSC. 

-  Subordinate  the  standard  C/SCSC 

clause  to  the  self  certification  clause, 
to  be  invoked  only  if  DCMC  and 
PMO  surveillance,  coupled  with 
review  of  the  Cost  Performance 
Report  (CPR),  suggests  faulty 

certification. 

This  is  a  unilateral 

determination  on  the  part  of 
the  government. 

2.  Objective:  Reduce  the  review  burden  on 
both  the  government  and  contractor,  and 
emphasize  the  presence  of  value  in  the 

management  systems,  rather  than  the  absence 
of  deficiencies. 

•  Initiate  a  Memorandum  of  Agreement 
with  the  DPRO  ensuring  regular 
surveillance  of  the  "certified" 
management  systems. 

•  Replace  the  buying  activity's  normal 

C/SCSC  review(s)  with  (a)  assistance 
visits  to  the  DPRO  to  ensure 
establishment  of  an  effective  surveillance 
program,  and  (b)  participation  in  DPRO 
surveillance  reviews. 

-  Deficiencies  identified  by  joint 
DPRO/buying  activity  surveillance 
reviews  to  be  evaluated  against  the 
CPR  to  determine  if  the  deficiency 


109 


has  a  material  impact  on  the  quality, 
timeliness,  or  utility  of  the  reported 
data. 

—  If  so,  consider  appropriate 
corrective  action  to  include 
invoking  a  formal  C/SCSC 
review. 

—  If  not,  no  action  is 
required. 

3.  Objective:  Put  earned  value  in  its  proper 
context  as  an  integrating  tool  for  cost, 
schedule,  and  technical  management. 

•  The  C/SCSC  are  inherently  a 
management  requirement,  not  a  reporting 
requirement.  Consequently,  the  contract 
statement  of  work  (SOW)  should  include 
the  requirement  for  the  contractor  to 
manage  using  (among  other  things) 
earned  value.  The  SOW  should  also 
reflect  the  requirement  for  the  periodic 
(usually  monthly)  contractor/PMO 
reviews  to  include  discussion  of  technical 
and  schedule  problems  in  their  earned 
value  context. 

4.  Objective:  Limit  reporting  to  what  can 
and  will  be  effectively  used. 

•  Consider  how  the  PMO  is  (or  will  be) 
organized  to  manage  the  effort  and  tailor 
reporting  to  those  needs. 

-  Assign  CPR  analysis  responsibility 
to  the  cognizant  PMO  technical 
organizations. 

-  Change  the  traditional  "program 
control"  or  "business  management" 
staffs  CPR  analysis  responsibilities  to 
those  of  an  internal  PMO  staff 
oversight  activity,  charged  with 


ensuring  that  the  sum  of  the 
individual  technical  staff  analyses 
represents  a  reasonable  whole. 

-  Structure  CPR  formats  1&  2  to 
reflect  the  way  the  PMO  and 
contractor  view  the  effort. 

—  For  example,  if  the  PMO  is 
organized  by  integrated 
product  teams  (IPTs),  wants 
to  manage  cost,  schedule,  and 
technical  performance  by 
IPTs,  and  has  no  need  for  the 
"standard"  WBS  format,  don't 
buy  it  (Note:  this  does  not 
obviate  the  need  to  establish  a 
MIL- STD  88  IB  compliant 
WBS.  That  need  transcends 
this  issue,  and  is  beyond  the 
scope  of  this  discussion). 

—  Similarly,  if  the  contractor 
is  not  organized  in  a 
"traditional"  functional 

manner,  don't  ask  for  a 
functionally  formatted  CPR 
format  2.  The  contractor 
should  be  free  to  structure 
format  2  to  reflect  how  he  is 
organized  to  manage.  If  the 
PMO  has  no  interest  in  seeing 
the  contract  status  as  the 
contractor  views  it,  format  2 
should  be  eliminated.  If  the 
contractor  manages  and  views 
status  in  the  same  manner  the 
PMO  does  (e.g.,  IPTs),  there 
is  no  need  for  format  2. 

--  Consider  eliminating  CPR 
format  5's  written  variance 
analysis,  or  at  least  limiting  it 
to  the  most  significant 
variances  as  determined  by 
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the  contractor.  This  should 
allow  submission  of  the  CPR 
5-10  days  after  the  close  of 
the  "books".  Minimal 
recurring,  written  variance 
analyses,  can  be  augmented 
by  a  requirement  for  the 
contractor  to  provide 
variance  analyses  as  requested 
based  on  PMO  staff  review. 
These  can  be  written  or 
provided  at  the  monthly 
contractor/PMO  reviews  (see 
item  3  above). 

5.  Objective:  Ensure  comprehensive 

planning  (one  of  the  major  benefits  of 
C/SCSC),  and  common  understanding  of  the 
task  by  both  parties. 

•  Conduct  a  baseline  planning  review  as 
early  as  possible  after  contract  award. 

-  Surveys  have  shown  that  baselines 
are  established  within  six  months  of 
contract  award,  but  C/SCSC 
demonstration  or  subsequent 
application  reviews  occur  12  to  18 
months  after  contract  award. 
Management  of  the  contract  cannot 
wait  for  the  C/SCSC  review. 


-  The  baseline  planning  review  should 
be  PMO  led,  consisting  primarily  of 
PMO  technical  staff  with  buying 
activity  C/SCSC  staff  in  a  supporting 
role. 

--  Focus  is  on  government 
and  contractor  technical 
manager  counterparts 
reviewing  cost  account 
planning  to  ensure  total 
coverage  of  the  statement  of 
work,  proper  scheduling,  and 
adequate  resourcing. 

—  All  cost  accounts  should  be 
planned,  at  least  to  a 
summary  level,  to  the  end  of 
the  contract.  Any  cost 
accounts  that  cannot  be 
established  in  the  initial 
planning  effort  should  have 
the  critical  defining  event(s) 
necessary  for  planning 
identified  and  made  an  item  of 
continuing  management 
interest. 


Ill 
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ABSTRACT 

Three  case  studies  involving  the  cruise 
missile  Tomahawk  air  vehicle,  sustainer 
engine,  and  inertial  navigation  element 
examined  government  acquisition  strategy 
and  how  this  affected  program  cost, 
performance,  and  schedule  outcomes.  The 
government  acquisition  strategy  was  focused 
on  decreased  production  phase:  cost  and 
improved  performance  (reliability)  for  the  air 
vehicle;  decreased  schedule  length  and  risk 
for  the  sustainer  engine;  and  decreased 
schedule  length  and  risk  and  cost  for  the 
inertial  navigation  element;  and  was  largely 
successful  in  meeting  these  goals.  However, 
a  key  factor  that  impacted  the  government 
acquisition  strategy  in  these  subsystem 
procurements  was  the  degree  to  which  the 
government  controlled  data  rights  and  had  a 
viable  technical  data  package. 

INTRODUCTION 

An  important  part  of  the  process  of 
understanding  acquisition  management 
methods  is  through  the  accumulation  and 
analysis  of  experience  from  mature  projects, 
especially  if  those  projects  involved  unusual 
situations  or  innovative  management 
techniques. 

This  paper  documents  the  experience  from 
1977  to  mid-1988  of  one  such  activity:  the 


Joint  Cruise  Missiles  Project  (JCMP).  That 
project  involves  some  unusual  situations  and 
innovative  techniques  that  may  also  be 
applicable  to  future  projects: 

1. )  A  family  of  missiles  with  a  high  degree 
of  commonality  between  models  was 
developed  concurrently;  these  models 
incorporated  advanced  technology  and 
performance  capabilities. 

2. )  Prime  contractor  competition  was 
maintained  until  the  end  of  the  Air  Launched 
Cruise  Missile  (ALCM)  Full  Scale 
Development  (FSD),  a  rare  occurrence  in 
major  military  system  acquisition. 

3. )  An  unusually  high  degree  of  competitive 
dual  sourcing  was  initiated  by  the 
government  project  office  during  FSD  for 
introduction  in  the  production  phase.  The 
government  strategy  was  pursued  as  a 
possible  means  to  meet  the  ALCM 
deployment  schedule  (FI 07  engine  and 
Inertial  Navigation  Element  (INE)),  reduce 
cost  (INE  and  Sea  Launched  Cruise  Missile 
(SLCM)  and  Ground  Launched  Cruise 
Missile  (GLCM)  airframe)  and  increase 
performance  (SLCM  and  GLCM  airframe). 
The  government  strategy  has  also  resulted  in 
the  project  office  dealing  directly  with  an 
extensive  set  of  associate  contractors  rather 
than  a  single  prime  contractor. 
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Any  one  of  these  features  is  rare  among 
recent  and  current  military  system  acquisition 
projects,  and  the  combination  of  all  three 
makes  the  cruise  missile  project  unique. 

Three  cruise  missile  program  case  studies 
were  performed:  (1)  the  Tomahawk  (GLCM 
and  SLCM)  air  vehicle;  (2)  the  FI 07 
sustainer  engine  for  the  ALCM,  GLCM,  and 
SLCM;  and  (3)  the  INE  for  the  ALCM, 
GLCM  and  land-attack  SLCM.  Each 
subsystem  used  in  the  three  case  studies  is  of 
critical  importance  to  a  large  share  of  the 
first  generation  cruise  missiles  that  have  been 
deployed. 

The  events  examined  in  this  paper  are  mainly 
since  January,  1977  when  the  Joint  Cruise 
Missiles  Project  Office  (JCMPO)  was 
formed  following  the  Defense  Systems 
Acquisition  Review  Council  (DSARC)  II 
review.  Each  service  had  an  active  cruise 
missile  development  project  for  several  years 
before  1977.  The  earlier  period  is  briefly 
discussed  to  provide  some  historical 
continuity,  but  the  focus  of  the  paper  is  on 
the  FSD  and  production  phases.  Similarly,  a 
May  1988  cutoff  date  is  used  for  the  three 
cruise  missile  case  studies  (except  in  one 
instance). 

While  the  cruise  missile  Tomahawk  air 
vehicle,  sustainer  engine,  and  INE  case 
studies  are  fairly  thorough,  they  are  not 
intended  to  form  a  complete  and  detailed 
project  history. 

BACKGROUND 

The  first  generation  modem  cruise  missiles 
are  the:  (1)  Air  Force  ALCM,  which 
includes  the  Boeing  Aircraft  Company 
(Boeing)  airframe,  Williams  International 
Corporation  (Williams)  FI  07  sustainer 
engine,  and  the  Litton  Inertial  Navigation 
Element  (INE);  (2)  GLCM,  which  includes 


the  General  Dynamics/Convair  Aerospace 
Division  (General  Dynamics)  Tomahawk 
airframe,  FI  07  engine,  and  McDonnell 
Douglas  Corporation  (McDonnell  Douglas) 
guidance  subsystem;  and  (3)  SLCM 
(originally  known  as  the  Submarine 
Launched  Cruise  Missile),  which  includes  the 
Tomahawk  airframe,  FI  07  engine,  and 
McDonnell  Douglas  guidance  subsystem. 
(The  Tomahawk  is  a  generic  name 
representing  the  first  generation  GLCM  and 
SLCM.  FI 07  will  be  generically  used  to 
designate  the  ALCM  sustainer  engine  (F107) 
and  the  GLCM  and  SLCM  sustainer  engine 
(F407).  INE  will  be  generically  used  to 
designate  the  inertial  navigation  platform, 
digital  computer,  and  power  supply  for  the 
land-attack  Tomahawk  (GLCM  and  SLCM) 
and  Boeing  (ALCM)  cruise  missiles.  The 
resulting  components  are  actually  known  as 
an  Inertial  Navigation  Element  (INE)  for  Air 
Force  applications  and  Reference 
Measurement  Unit  and  Computer  (RMUC) 
for  Navy  land-attack  applications.) 

Numerous  cruise  missiles  had  previously 
been  developed,  and  in  some  cases  deployed, 
prior  to  the  ALCM,  GLCM,  and  SLCM. 
These  weapon  systems  included  the  Snark, 
Matador,  Hound  Dog,  and  Harpoon.  What 
sets  the  ALCM,  GLCM,  and  SLCM  apart 
from  these  earlier  cruise  missiles  is:  (1)  their 
compact  size,  which  permits  increased 
numbers  to  be  carried  per  platform,  in  part 
due  to  a  miniaturized  nuclear  warhead 
(ALCM,  GLCM,  and  nuclear-armed  land- 
attack  SLCM);  (2)  their  low  observables, 
which  increases  their  chances  of  reaching  the 
target,  due  to  their  low  radar  cross-section 
design,  small  size,  and  the  Williams  FI 07 
turbofan  engine  (with  its  reduced  infrared 
signature  versus  a  turbojet  engine);  (3) 
design  features  such  as  their  relatively  long 
range,  which  permits  additional  targets  to  be 
attacked,  due  to  the  highly  efficient  FI 07 
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engine;  and  (4)  their  high  accuracy,  which 
increases  the  chances  of  target  kill,  due  to 
the  McDonnell  Douglas  guidance  system 
which  incorporates  in-flight  terrain  contour 
matching  updates  to  improve  accuracy.  In 
general,  earlier  cruise  missiles  were  lacking 
in  each  of  these  areas.  (For  the  remainder  of 
this  paper,  the  ALCM,  GLCM,  and  SLCM 
are  referred  to  as  first  generation  cruise 
missiles.) 

TOMAHAWK  AIR  VEHICLE  CASE 
STUDY 

As  previously  mentioned,  Boeing  and 
General  Dynamics  were  the  developers  of 
the  ALCM  air  vehicle  and  GLCM  and 
SLCM  airframe,  respectively.  President 
Jimmy  Carter's  cancellation  of  the  B-1A 
bomber  program  on  June  30,  1977  in  favor 
of  the  long-range  ALCM  carried  by  the  B-52 
bomber  elevated  the  ALCM  development 
program  to  a  high  level  of  national 
importance.  The  government  established  a 
FSD  competitive  flyoff  between  the  Boeing 
and  General  Dynamics  cruise  missiles  to 
establish  the  best  design.  Had  General 
Dynamics  won  the  ALCM  competition,  the 
government  would  likely  have  made  Boeing 
the  production  second  source  for  the  ALCM 
airframe,  and  possibly  for  the  GLCM  and 
SLCM  airframes,  thus  forcing  production 
phase  competition  between  General 
Dynamics  and  Boeing.  However,  Boeing 
was  declared  the  ALCM  competition  winner 
on  March  25,  1980,  and  on  April  30,  1980  it 
was  made  the  sole  ALCM  airframe 
production  supplier. 

This  left  General  Dynamics  and  McDonnell 
Douglas  as  the  sole  production  sources  for 
the  Tomahawk  airframe  and  guidance 
subsystems,  respectively.  The  government 
was  concerned  about  General  Dynamics'  cost 
inefficiency  and  what  it  believed  to  be  an 


excessive  short-term  profit  orientation.  In 
addition,  the  government  was  concerned 
about  inadequate  missile  airframe  reliability, 
which  directly  affects  performance,  and 
could  adversely  impact  production  delivery 
schedules.  For  example,  by  August  1981  the 
General  Dynamics'  Tomahawk  airframe  had 
exhibited  a  pattern  of  flight  test  failures 
related  to  quality  assurance  problems 
followed  by  close  government  scrutiny  that 
resulted  in  flight  test  successes.  (These 
cyclical  sequences  demonstrated  that  the 
problems  were  correctable  and  partially  due 
to  insufficient  manufacturing  discipline  by 
General  Dynamics;  these  failures  led,  on  June 
22,  1982,  to  the  most  severe  censure  ever 
issued  against  a  defense  contractor  for 
quality  assurance  problems  [1].)  The 
government  was  also  concerned  about 
McDonnell  Douglas  cost  inefficiency  and 
profit  maximizing  behavior  in  supplying  the 
guidance  subsystem. 

The  initial  Tomahawk  air  vehicle  cost, 
performance,  and  schedule  solution  point 
was  unacceptable  to  the  government,  given 
the  high  air  vehicle  cost  and  that  General 
Dynamics'  airframe  reliability  problems  left 
performance  below  the  minimum  required  by 
the  government. 

The  government's  main  desire  was  to 
decrease  Tomahawk  production  cost  while 
increasing  performance  (reliability).  General 
Dynamics  and  McDonnell  Douglas  did  not 
have  a  strong  incentive  to  correct  the 
problems  while  in  a  sole  source  mode.  Both 
were  powerful  defense  contractors  and 
operating  with  undefinitized  letter  contracts 
since  1976,  which  gave  them  an  advantage 
over  the  government  in  justifying  high 
program  costs  and  increased  profits.  As  a 
result,  the  government  considered  the 
possibility  of  dual  sourcing  the  Tomahawk 
airframe  and  guidance  subsystem. 
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General  Dynamics  initially  resisted 
government  attempts  to  second  source  the 
Tomahawk  airframe.  It  also  wanted 
McDonnell  Douglas  as  a  subcontractor  for 
the  guidance  subsystem,  which  would  have 
increased  General  Dynamics'  cost  (thus 
potentially  lowering  its  overhead  rate)  and 
profit,  because  profits  were  calculated  in  part 
as  a  markup  over  costs.  However,  by  mid- 
1981,  McDonnell  Douglas  had  more 
incentive  to  accept  a  dual  sourcing 
arrangement  than  General  Dynamics  because 
the  Litton  INE  portion  of  the  guidance 
system  was  separately  contracted  to  the 
government  on  February  27,  1981  rather 
than  subcontracted  to  McDonnell  Douglas  (it 
was  about  65  percent  of  the  total  McDonnell 
Douglas  guidance  system  contract  value), 
thus  considerably  reducing  McDonnell 
Douglas  potential  contract  dollar  amount  and 
profit. 

The  government  leveraged  General 
Dynamics  to  participate  in  Tomahawk  dual 
sourcing  in  late  1981  through  two  actions. 
First,  it  invited  key  McDonnell  Douglas 
personnel  to  the  General  Dynamics'  facility 
for  familiarization.  Second,  it  informed 
General  Dynamics  on  November  12,  1981  of 
the  advantages  and  disadvantages  of  not 
participating  in  a  dual  sourcing  agreement. 
The  negative  consequences  of  General 
Dynamics  nonparticipating  outweighed  the 
benefits.  These  two  actions  were  the  turning 
point  in  gaining  General  Dynamics' 
cooperation  to  develop  a  suitable  second 
source  plan,  as  General  Dynamics'  apparently 
realized  that  the  government  was  serious  in 
adopting  the  dual  source  acquisition  strategy. 

The  government  reviewed  four  possible 
acquisition  strategies  to  obtain  a  more 
satisfactory  outcome;  they  all  involved  a 
second  production  source  and  some  form  of 
competition.  These  options  involved 
procurement  by:  (1)  a  second  producer 


using  the  General  Dynamics'  airframe 
technical  data  package  as  the  principle 
medium  for  transferring  technology,  (2) 
airframe  alternate  design,  (3)  identical  design 
airframe-only  dual  sourcing  using  a 
Leader/Follower  procurement  strategy,  and 
(4)  identical  design  air  vehicle  (airframe  and 
guidance  system  less  FI 07  engine)  dual 
sourcing  using  a  Leader/Follower 
procurement  strategy.  None  of  these  options 
would  provide  the  government  with  a  more 
competitive  cost,  performance,  and  schedule 
situation  during  cruise  missile  development, 
but  rather,  were  oriented  towards  production 
improvements.  These  options  were  part  of 
the  government  strategy  to  achieve  higher 
airframe  reliability  and  lower  air  vehicle 
production  costs  while  still  meeting 
production  delivery  schedules. 

Although  the  government  had  paid  for  a 
Tomahawk  airframe  data  package,  it  was  not 
thorough  enough  to  permit  another 
contractor  to  fabricate  the  airframe, 
particularly  given  that  cruise  missile 
production  techniques  employed  precision 
machining  and  welding  rather  than  the  more 
traditional  fabrication  method  using  rivets. 
The  government  faced  the  possibility  with 
the  technical  data  package  option  of  a 
considerable  development  cost  increase, 
coupled  with  only  limited  production  cost 
reduction.  However,  some  potential 
improvement  in  reliability  would  likely  result. 

An  alternate  design  airframe  suitable  for  sea, 
and  particularly  submarine,  launch  would 
have  represented  a  major  development 
activity  due  to  launch  and  storage 
environment  considerations.  This  approach 
would  have  resulted  in  increased 
development  cost  and  schedule,  with  only 
limited  production  cost  reduction  likely. 
However,  some  potential  improvement  in 
reliability  would  probably  result  with  this 
second  source  option. 
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The  airframe  identical  design 
Leader/Follower  approach  would  have 
involved  selecting  a  second  contractor  to 
fabricate  the  Tomahawk  airframe.  A 
Leader/Follower  arrangement  differs  from  a 
technical  data  package  approach  in  that  the 
Leader  actually  participates  in  the  technology 
transfer  process,  which  involves  considerable 
effort  on  the  part  of  the  Leader.  With  this 
option  the  government  would  have  faced 
increased  development  cost  for  technology 
transfer  between  General  Dynamics  and  the 
second  contractor,  with  only  moderate 
production  cost  reduction  likely.  However, 
some  potential  improvement  in  reliability 
would  probably  result. 

The  air  vehicle  identical  design 
Leader/Follower  option,  which  was  actually 
implemented,  involved  selecting  a  second 
contractor  to  fabricate  both  the  Tomahawk 
airframe  and  guidance  subsystems.  (In  this 
case  the  FI 07  engine  was  provided  by  the 
government  to  each  contractor.)  General 
Dynamics  and  McDonnell  Douglas  were 
chosen  to  be  the  Followers  on  each  other's 
guidance  and  airframe  subsystems, 
respectively.  The  government  proposed  to 
General  Dynamics  and  McDonnell  Douglas  a 
series  of  measures  to  efficiently  carry  out  the 
technology  transfer  process  and  reduce  the 
level  of  government  up-front  funding 
necessary  to  implement  dual  Tomahawk 
production  sources.  It  also  provided  the 
companies  an  acceptable  assurance  that  in 
the  event  of  an  early  Tomahawk  program 
cancellation,  costs  incurred  by  the 
contractors  technology  transfer,  tooling,  and 
test  equipment  would  be  covered  by 
cancellation  payments. 

The  air  vehicle  identical  design 
Leader/Follower  dual  source  option  had  the 
potential  to  yield  considerable  production 
cost  savings  due  to  the  intense  rivalry 
between  General  Dynamics  and  McDonnell 


Douglas  in  general  as  well  as  specifically  for 
cruise  missile  business.  Production  reliability 
improvements  were  also  likely  because  of  the 
competitive  nature  of  the  government 
acquisition  strategy.  Not  only  did  a  yearly 
price  competition  exist,  but  other  factors, 
including  quality  assurance  and  the  ability  to 
meet  delivery  schedules,  were  part  of  the 
source  selection  criteria.  The  air  vehicle 
identical  design  Leader/Follower  dual  source 
option  also  had  the  potential  to  provide  the 
shortest  Follower  initial  delivery  time  due  to 
the  non-hostile  nature  of  the  technology 
transfer  process.  In  fact,  only  the  air  vehicle 
identical  design  Leader/Follower  dual  source 
option  eliminated  the  somewhat  complicated 
contractual  problem  of  selecting  and 
establishing  the  second  source,  since  General 
Dynamics  and  McDonnell  Douglas  became 
the  Follower  on  each  other's  Tomahawk 
subsystem. 

The  air  vehicle  identical  design 
Leader/Follower  option  was  selected  by  the 
JCMPO,  and  approved  by  the  Department  of 
the  Navy  on  January  18,  1982.  The 
government  justification  used  to  gain 
approval  to  implement  this  dual  source 
approach  was  that  it  would  increase  the 
industrial  mobilization  base  compared  to  the 
other  dual  sourcing  options.  While  the 
justification  was  needed  to  gain  higher  level 
government  approval,  the  program  office's 
main  goal  was  to  decrease  production  cost, 
improve  performance,  and  meet  the  delivery 
schedule. 

The  government  acquisition  approach, 
coupled  with  a  large  SLCM  procurement 
quantity  increase  in  December  1981  (3994 
versus  644  in  March  1981),  made  dual 
sourcing  the  Tomahawk  air  vehicle  a  viable 
option  for  all  parties.  The  government's  dual 
sourcing  plan  was  accepted  by  General 
Dynamics  and  McDonnell  Douglas  in  FY82 
and  the  first  year  of  head-to-head 
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competition  between  contractors  was  in 
FY85. 

From  FY85  through  FY88  General 
Dynamics  won  three  of  the  four  annual  split- 
buys  (FY85,  FY86,  and  FY88)  against 
McDonnell  Douglas  and  an  average  quantity 
of  58  versus  42  percent.  McDonnell 
Douglas  won  annual  split-buys  in  FY89, 
FY91,  and  FY92  (while  General  Dynamics 
won  in  FY90  and  the  FY91  supplemental 
buy).  The  ability  of  McDonnell  Douglas 
(Follower)  to  win  four  of  the  nine  annual 
split-buys  through  FY92,  coupled  with  a 
strong  competition  in  FY87  through  FY92, 
indicates  that  the  Tomahawk  air  vehicle 
production  phase  dual  sourcing  competition 
has  indeed  been  viable. 

The  outcomes  through  FY88  of  the  air 
vehicle  identical  design  Leader/Follower 
acquisition  approach  include:  (1)  production 
cost  savings,  estimated  to  be  several  hundred 
million  dollars  or  more,  over  that  projected 
for  both  companies  operating  in  a  sole 
source  role;  (2)  a  reduction  in  quality 
assurance  problems;  and  (3)  the  ability  to 
meet  production  delivery  schedules.  For 
example,  the  JCMPO  estimated  a  $0.46 
million  (FY87)  decrease  in  air  vehicle 
average  unit  cost  for  the  FY88  buy  versus 
sole  source  projections  [2],  Given  the  FY88 
buy  of  475  air  vehicles,  a  substantial  savings 
likely  occurred  in  this  year  alone. 

The  shift  in  the  outcomes  from  the  original 
sole  source  production  plan  is  largely  tied  to 
the  government  acquisition  strategy  which 
provided  an  incentive  to  both  contractors  to 
reduce  inefficiency  through  the  use  of 
competitive  annual  missile  buys.  It  is  also 
the  result  of  an  existing  rivalry  between 
General  Dynamics  and  McDonnell  Douglas 
that  has  led  to  a  strong  price  competition  on 
the  Tomahawk  production  program. 


SUSTAINER  ENGINE  CASE  STUDY 

Williams  International  Corporation  (WIC) 
was  selected  by  the  government  as  the  Air 
Launched  Cruise  Missile  (ALCM)  and  Sea 
Launched  Cruise  Missile  (SLCM)  sustainer 
engine  contractor  in  mid- 1973  and  May 
1976,  respectively.  This  relationship  was 
extended  to  the  Ground  Launched  Cruise 
Missile  (GLCM)  program  that  was  initiated 
in  January  1977. 

Although  there  was  no  reason  to  doubt  the 
design  or  reliability  of  the  FI 07  sustainer 
engine,  WIC  was,  in  early  1977,  a  small 
privately  owned  corporation  with  no  high 
rate  production  experience  and  whose 
maximum  business  was  less  than  $40  million 
per  year.  Engine  fabrication  times  during  the 
FSD  program  phase  were  far  too  long  to 
support  planned  production  delivery 

schedules.  For  example,  it  initially  took  3 
months  to  make  one  F107  engine  combustor 
versus  the  potential  need  of  fabricating  up  to 
5  to  10  per  day  during  production  [3],  The 
government  was  quite  risk  averse  to  any 
possible  ALCM  production  schedule 

slippage  because  of  the  increased  importance 
of  the  ALCM  once  the  B-1A  was  canceled 
by  President  Carter.  The  government  was 
interested  in  reducing  the  potential 

production  delivery  schedule  length. 

The  government  also  regarded  the  potential 
WIC  FI 07  engine  production  cost  as  high, 
and  indicative  of  inefficient  contractor 
operation.  However,  reducing  cost  was  of 
less  concern  during  1977-1978  than 
achieving  an  acceptable  production 
fabrication  capability. 

The  initial  FI 07  engine  cost,  performance, 
and  schedule  solution  point  was 
unacceptable  to  the  government,  since  the 
initial  WIC  FI 07  fabrication  rate  was  far 
below  the  minimum  required  by  the 
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government.  In  effect,  the  initial  FI 07 
delivery  schedule  was  greater  than  the 
maximum  allowable  to  meet  the  ALCM 
delivery  schedule.  Consequently,  the 

government's  main  desire  was  to  decrease 
production  schedule  length  and  risk,  maintain 
demonstrated  performance  levels,  and  reduce 
production  cost,  if  possible. 

The  government  reviewed  four  possible 
options  to  obtain  a  more  satisfactory 
outcome;  they  all  involved  some  form  of 
competition.  These  options  involved 
procurement  by:  (1)  engine  technical  data 
package,  (2)  engine  alternate  design,  (3) 
identical  design  engine  dual  sourcing  using  a 
Leader/Follower  procurement  strategy,  and 
(4)  development  of  second  sources  for  all 
critical  or  major  engine  subassemblies  or 
parts  except  for  the  key  fuel  control  unit  (the 
single  most  expensive  FI  07  engine 
component).  Only  option  4  would  provide 
the  government  with  a  more  desirable  cost, 
performance,  and  schedule  solution  point 
during  engine  development,  but  it  was 
insufficient  as  a  stand  alone  option  to  greatly 
reduce  the  production  schedule  length. 
Conversely,  the  other  options  were  mainly 
oriented  towards  potential  production 
schedule  improvements. 

The  acquisition  strategy  considered  by  the 
government  was  clouded  by  the  WIC  claim 
during  1977  that  it  held  proprietary  data 
rights  on  145  F107  engine  components. 
Although  the  burden  of  proof  for  rights  to 
data  would  have  been  on  WIC,  the  lengthy 
litigation  process  could  have  delayed  the 
critical  ALCM  delivery  schedule. 

Because  of  the  WIC  claims,  purchase  of  an 
FI 07  engine  technical  data  package  would 
have  been  expensive  for  the  government,  and 
would  not  have  reduced  the  government's 
concern  over  production  schedule  length. 
The  schedule  concern  was  due  to  the 


complex  fabrication  process  and  tolerances 
employed  in  the  FI 07  engine,  and  the 
possibility  that  not  all  necessary  production 
knowledge  would  be  included  or  successfully 
transferred  in  the  technical  data  package. 
With  the  technical  data  package  option  the 
government  faced  the  possibility  of 
considerable  up-front  cost.  In  addition,  if  the 
second  source  was  unable  to  assimilate  the 
technical  data  package,  then  increased 
production  cost  and  schedule  length  and 
decreased  performance  were  possible. 

An  alternate  design  sustainer  engine  would 
have  represented  a  major  development 
activity  due  to  the  technical  complexity  of 
engine,  thus  increasing  development  cost  and 
schedule.  In  addition,  it  may  not  have  been 
capable  of  achieving  the  desired  production 
delivery  schedule,  at  least  in  the  near-term, 
given  that  the  time  to  qualify  such  an  engine 
would  have  been  several  years.  This  long  a 
qualification  time  would  also  have  given 
WIC  a  considerable  price  competition 
advantage  over  the  alternate  design  engine, 
and  thus,  only  limited  production  cost 
reduction,  if  any,  might  result  (versus  WIC 
sole  source  production).  Finally,  potential 
performance  improvements  from  the 
alternate  design  engine  were  estimated  to 
range  from  zero  to  only  a  few  percent. 
Given  that  the  government  viewed  FI 07 
engine  performance  as  satisfactory,  an 
alternate  design  engine  with,  at  best,  only  a 
marginal  increase  in  performance  did  not 
present  a  convincing  argument  for  selection 
on  technical  grounds. 

The  identical  design  Leader/Follower 
approach  would  have  involved  selecting  a 
second  contractor  to  fabricate  the  FI 07 
engine.  Because  of  WIC's  claim  to  data 
rights,  the  government  would  have  incurred 
considerable  up-front  cost  for  technology 
transfer  between  WIC  and  the  second 
contractor,  with  only  limited  to  moderate 
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production  cost  reduction.  The  identical 
design  Leader/Follower  option  would  have 
likely  provided  the  desired  decrease  in 
production  FI 07  engine  delivery  schedule, 
while  maintaining  engine  performance 
achieved  by  1977.  The  alternate 
subassembly  and  parts  suppliers  option  was 
not  viewed  as  a  sufficient  stand  alone 
approach  to  achieve  the  desired  production 
engine  delivery  schedule,  but  rather  as  a 
means  of  augmenting  one  of  the  other 
options  to  reduce  FI 07  engine  production 
risk. 

The  alternate  subassembly  and  parts 
suppliers  option  was  chosen  on  November  2, 
1977,  while  a  modified  form  of  the  sustainer 
engine  identical  design  Leader/Follower 
option,  was  selected  by  the  government  in 
March  1978.  The  modified  Leader/Follower 
option  kept  the  Follower  as  a  subcontractor 
to  WIC  for  the  duration  of  the  engine  buy. 
(Separately  contracting  with  the  Follower 
was  possible,  but  would  have  required 
royalty  payments  be  made  to  WIC.) 

WIC  initially  attempted  to  pick  a  Follower 
with  limited  applicable  production 
experience  to  prevent  a  successful 
production  price  competition.  The 
government  did  not  approve  the  WIC 
recommended  contractor  and  suggested  that 
a  contractor  be  named  with  considerable 
high  rate  production  experience  to  minimize 
potential  schedule  risk.  WIC  then 
reconsidered  its  Follower  selection  and 
proposed  Teledyne  Corporation,  Continental 
Aircraft  Engine  Division  (TCAE)  on 
September  18,  1978.  TCAE  was  acceptable 
to  the  government  since  they  were 
competitively  developing  the  FI 06  engine 
against  WIC  in  the  early  phase  of  the  SLCM 
program  and  had  high  rate,  small  engine 
production  experience  with  the  Harpoon 
F402  turbojet  engine. 


Given  that  TCAE  was  to  remain  a 
subcontractor  to  WIC,  the  up-front  cost  to 
the  government  was  considerably  less  than  if 
it  had  a  direct  contracting  relationship  with 
the  government.  However,  the  reduced  up¬ 
front  cost  must  be  balanced  against  the 
potentially  higher  production  cost  with  this 
approach  since  WIC  was  guaranteed  a 
monthly  minimum  buy  and  TCAE  was 
limited  to  a  monthly  maximum  buy.  The 
modified  Leader/Follower  option  also  had 
the  best  chance  of  meeting  the  necessary 
production  delivery  schedule,  while 
maintaining  the  acceptable  FI 07  engine 
performance  that  had  already  been  achieved. 

The  outcomes  through  FY88  of  the  identical 
design  Leader/Follower  procurement 
approach  include:  (1)  meeting  the 

production  delivery  schedules  for  the 
ALCM,  GLCM,  and  SLCM;  (2)  reducing 
FI 07  engine  cost  as  compared  to  cost  in  the 
FSD  phase;  (3)  and  maintaining  acceptable 
FI 07  engine  performance. 

The  shift  in  the  cost,  performance,  and 
schedule  solution  point  is  largely  tied  to  the 
government  acquisition  strategy  which 
established  a  second  FI 07  engine  production 
source.  Although  the  technology  transfer 
process  between  WIC  and  TCAE  was  not 
initially  without  problems,  it  forced  WIC  to 
closely  examine  and  "rethink"  the  engine 
fabrication  process.  As  a  result,  WIC 
become  a  more  efficient  and  lower  cost 
turbofan  engine  manufacturer.  This 
benefited  the  government  by  providing  the 
desired  reduction  in  engine  fabrication  time, 
as  well  as  some  decrease  in  cost. 

It  is  difficult  to  say  that  the  government 
Leader/Follower  dual  source  acquisition 
strategy  could  be  justified  on  a  cost  basis, 
since  the  retention  of  TCAE  as  a 
subcontractor  to  WIC,  coupled  with  the 
monthly  WIC  minimum  and  TCAE 
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maximum  buys,  guaranteed  WIC  getting  the 
majority  of  each  years  split-buy  and 
effectively  eliminated  an  open  price 
competition.  The  key  to  the  limitation  was 
the  data  rights  issue.  By  1980  it  had  been 
estimated  by  the  government  that  the  validity 
of  WIC's  data  rights  claims  on  the  145  FI 07 
engine  components  had  greatly  eroded,  due 
to  patents  expiring  and  invalid  claims  against 
other  components.  Unfortunately  for  the 
government,  the  weak  WIC  position  was  not 
known  until  almost  1  1/2  to  2  years  after 
signing  the  dual  sourcing  arrangement  with 
WIC  in  1978.  Consequently,  potential  cost 
savings  that  may  have  occurred  through  an 
open  production  dual  source  competition 
was  not  possible  in  the  FI 07  engine  case. 

INERTIAL  NAVIGATION  ELEMENT 
CASE  STUDY 

Litton  Guidance  and  Control  Systems 
Division  (Litton)  was  selected  by  Boeing 
Aerospace  Company  as  the  Air  Launched 
Cruise  Missile  (ALCM)  inertial  navigation 
element  (INE)  contractor  in  mid- 1973.  In 
October  1975  McDonnell  Douglas 
Astronautics  Company  was  selected  as  the 
Sea  Launched  Cruise  Missile  (SLCM) 
guidance  system  contractor,  with  Litton  as 
its  INE  subcontractor.  The  McDonnell 
Douglas/Litton  relationship  was  extended  to 
the  Ground  Launched  Cruise  Missile 
(GLCM)  program  that  was  initiated  in 
January  1977. 

There  was  little  reason  to  doubt  the  design, 
performance,  or  reliability  of  the  Litton  INE 
since  it  had  already  been  extensively  flight 
tested,  and  a  closely  related  INE  was 
developed  and  satisfactorily  tested,  in  the  F- 
15  aircraft  by  January  1977.  However,  as  in 
the  FI 07  engine  case,  the  government  was 
interested  in  reducing  the  production 


schedule  length  and  risk  in  order  to  meet  the 
ALCM  delivery  schedule. 

The  government  also  regarded  the  Litton 
INE  production  cost  estimates  as  moderate 
to  high,  and  indicative  of  inefficient 
contractor  operation.  Given  the  government 
experience  in  reaching  the  FI 07  engine  dual 
source  agreement,  in  March  1978,  it  also 
considered  the  possibility  that  an  INE 
production  competition  could  result  in  cost 
savings. 

The  initial  INE  cost,  performance,  and 
schedule  solution  point  was  unacceptable  to 
the  government,  since  the  initial  INE 
fabrication  rate  was  likely  below  the 
minimum  required  by  the  government.  In 
effect,  the  initial  INE  delivery  schedule  was 
probably  greater  than  the  maximum 
allowable  to  meet  the  ALCM  delivery 
schedule.  In  addition,  the  initial  INE  cost 
was  likely  near  the  maximum  government 
level.  Consequently,  the  government's  main 
desire  was  to  decrease  production  schedule 
length,  maintain  demonstrated  performance 
levels,  and  reduce  production  cost. 

The  government  reviewed  four  possible 
options  to  obtain  a  more  satisfactory 
outcome;  they  all  involved  a  second 
production  source  and  some  form  of 
competition.  These  options  involved 
procurement  by:  (1)  INE  technical  data 
package,  (2)  INE  directed  licensing,  (3)  INE 
alternate  design,  and  (4)  identical  design 
engine  dual  sourcing  using  a 
Leader/Follower  procurement  strategy. 

The  acquisition  strategy  considered  by  the 
government,  as  in  the  FI 07  engine  case,  was 
clouded  by  the  Litton  claim  to  data  rights  on 
15  or  more  key  INE  components  [4],  In  the 
INE  case,  McDonnell  Douglas  and  the 
government  realized  that  the  Litton  claim  to 
data  rights  was  likely  valid. 
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Because  of  the  Litton  claim,  purchase  of  a 
INE  technical  data  package  would  have  been 
expensive  for  the  government,  and  would  not 
have  reduced  the  government's  production 
delivery  schedule  concern.  The  concern  was 
due  to  the  complex  fabrication  process  and 
tolerances  present  in  the  key  INE 
components  (e.g.,  gyroscope  and 
accelerometer),  and  the  possibility  that  not 
all  necessary  production  knowledge  would 
be  included  or  successfully  transferred  in  the 
technical  data  package.  The  government 
would  have  had  to  pay  a  considerable  up¬ 
front  cost  to  Litton  for  the  technical  data 
package.  In  addition,  if  the  second  source 
was  unable  to  assimilate  the  technical  data 
package,  then  increased  production  cost  and 
schedule  length  and  decreased  performance 
were  possible. 

Litton  was  unwilling  to  license  another 
contractor  to  fabricate  key  INE  components 
(e.g.,  gyroscope  and  accelerometer)  unless  a 
substantial  fee  was  paid.  However,  this 
acquisition  approach  would  also  not  have 
reduced  the  government's  production 
delivery  schedule  concern  for  the  same 
reasons  as  given  for  the  technical  data 
package.  As  in  the  technical  data  package 
case,  the  government  would  have  had  to  pay 
a  considerable  up-front  cost  to  Litton  to 
proceed  with  the  licensing  option. 

An  alternate  design  INE  represented  a 
moderately  complex  development  activity 
due  to  component  development  as  well  as 
subsystem  integration  that  would  be 
necessary,  thus  increasing  development  cost 
and  schedule.  In  addition,  it  may  not  have 
been  capable  of  achieving  the  desired 
delivery  schedule  in  the  early  portions  of  the 
ALCM  production  phase.  The  added  time  to 
qualify  the  Follower  would  also  have  given 
Litton  a  considerable  price  competition 
advantage  over  the  alternate  design  INE,  and 
thus,  only  limited  production  cost  reduction 


might  result  (versus  Litton  sole  source 
production).  Finally,  potential  performance 
improvements  from  the  alternate  design 
engine  were  estimated  to  be  non-existent. 
Given  that  the  government  viewed  the  Litton 
INE  performance  as  satisfactory,  an  alternate 
design  INE  with,  at  best,  the  same 
performance  did  not  present  an  argument  for 
selection  on  technical  grounds. 

The  identical  design  Leader/Follower 
approach  would  have  involved  selecting  a 
second  contractor  to  fabricate  the  INE. 
Because  of  Litton's  claim  to  data  rights, 
McDonnell  Douglas  and  the  government 
were  prohibited  by  Litton  to  implement  this 
approach  in  an  unmodified  form.  The 
identical  design  Leader/Follower  option 
would  have  likely  provided  the  desired 
decrease  in  the  production  INE  delivery 
schedule,  while  maintaining  INE 
performance  achieved  through  1978, 

Litton  stated  in  May  1978  that  the  key  to  a 
successful  technology  transfer  was  the  ability 
to  transfer  processes  and  know-how,  not 
products,  and  that  they  had  done  it 
successfully  36  times  in  the  past  with  other 
Litton  divisions  [5],  Litton  proposed  Litton 
Systems  Limited  (LSL)  as  the  potential  INE 
Follower  in  June  1978,  and  stated  that  there 
had  been  effective  head  to  head  competition 
between  these  two  divisions  seven  times  in 
the  past.  Both  McDonnell  Douglas  and  the 
government  recognized  that  the  Litton 
proposed  dual  sourcing  approach  provided 
the  lowest  risk  and  up-front  cost  to  the 
government,  the  ability  to  meet  production 
delivery  schedules,  and  the  potential  for 
production  cost  savings. 

The  Litton  proposed  dual  sourcing  approach 
was  selected  by  McDonnell  Douglas  and  the 
government  on  October  13,  1978.  It 
involved  maintaining  Litton  as  the  Leader, 
selecting  LSL  as  an  independent  contractor 
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as  well  as  the  Follower  to  fabricate  the  INE, 
and  keeping  both  Litton  divisions  as  a 
subcontractor  to  McDonnell  Douglas  (for 
the  GLCM  and  SLCM).  Other  provisions  of 
the  technology  transfer  agreement  between 
Litton  and  LSL  included  the  waiving  of 
royalty  charges  and  license  fees,  a  capital 
investment  incentive  clause  to  purchase 
necessary  equipment  to  achieve  rate 
production,  a  profit  level  cap  depending 
upon  contract  type,  minimum  production 
quantity  awards  for  each  Litton  division 
based  upon  the  annual  production  buy,  and 
the  ability  for  the  government  to  purchase  all 
INEs  from  a  single  Litton  division  if  the 
other  division's  performance  or  price  was  not 
acceptable. 

The  terms  of  the  agreement  were  more 
favorable  to  the  government  on  a  point  by 
point  basis  than  those  reached  with  WIC  for 
the  FI 07  engine  dual  sourcing  arrangement. 
For  example,  while  a  monthly  minimum  buy 
was  established  for  each  Litton  division,  no 
monthly  maximum  was  included,  and  thus  a 
greater  chance  existed  for  an  open  price 
competition  between  the  two  suppliers  than 
in  the  WIC  and  TCAE  sustainer  engine  case 
discussed  (all  other  things  held  constant). 

Given  the  use  of  LSL  as  the  Follower, 
coupled  with  a  capital  investment  incentive 
clause,  the  up-front  cost  to  the  government 
was  minimal  and  considerably  less  than  if  any 
other  option  had  been  chosen.  The  identical 
design  Leader/Follower  approach  also 
potentially  had  the  lowest  schedule  risk, 
including  meeting  the  necessary  production 
delivery  schedule,  while  maintaining  the 
acceptable  INE  performance  that  had  already 
been  achieved. 

On  February  27,  1981,  the  government 
broke-out  the  INE  from  McDonnell  Douglas 
and  contracted  directly  with  Litton  and  LSL 
in  an  attempt  to  further  reduce  potential 


production  costs.  The  first  year  of  head  to 
head  competition  between  the  two  Litton 
divisions  was  FY81.  From  FY81  through 
FY88,  Litton  and  LSL  won  2  and  6  of  the  8 
total  competitive  split-buys,  respectively. 
(The  average  quantity  won  by  Litton  and 
LSL  through  FY88  was  42  and  58  percent, 
respectively.)  The  Follower,  LSL,  won  the 
majority  of  the  annual  split-buys  from  Litton, 
the  INE  developer,  from  FY81  through 
FY88.  LSL  won  the  first  four  annual  split- 
buys,  but  during  FY85  through  FY88, 
neither  Litton  division  won  two  consecutive 
annual  competitions.  The  competent 
Follower  showing  in  winning  the  majority  of 
the  split-buys,  coupled  with  a  strong 
competition  in  FY85  through  FY88  is 
indicative  of  a  vigorous  production  dual 
source  competition. 

The  outcomes  through  FY88  of  the  identical 
design  Leader/Follower  procurement 
approach  include:  (1)  meeting  the 

production  delivery  schedules  for  the 
ALCM,  GLCM,  and  SLCM;  (2)  a  reduction 
in  INE  cost  in  the  production  versus  the 
development  phase;  (3)  likely  production 
cost  savings  versus  keeping  Litton  in  a  sole 
source  role;  and  (4)  maintaining  acceptable 
INE  performance. 

The  shift  in  the  cost,  performance,  and 
schedule  solution  point  is  largely  tied  to  the 
government  acquisition  strategy  which 
established  LSL  as  the  second  INE 
production  source.  It  is  also  related  to  the 
much  more  favorable  technology  transfer 
terms  the  government  had  between  Litton 
and  LSL  versus  WIC  and  TCAE  in  the  F107 
engine  case,  coupled  with  a  strong 
production  phase  dual  source  competition 
between  the  Litton  divisions  (as  discussed 
above). 

CONCLUSIONS 
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The  three  cruise  missile  cases  evaluated  here 
represent  a  broad  range  of  programmatic 
conditions  for  the  government  and 
contractors.  In  each  case,  the  government 
used  production  phase  dual,  sourcing  as  a 
means  of  reducing  one  or  more  undesirable 
cost,  performance,  or  schedule 
characteristics  present  in  the  development 
phase.  However,  the  production  phase 
outcomes  for  the  three  subsystems  were 
largely  dependent  upon  the  FSD  "starting 
point"  between  the  government  and 
contractors. 

The  primary  government  objectives  were 
different  for  each  production  phase 
competition.  For  the  sustainer  engine,  the 
government  attempted  to  reduce  fabrication 
schedule  length  and  risk;  for  the  INE,  reduce 
fabrication  schedule  length  and  risk,  and 
reduce  cost;  and  for  the  AUR,  reduce  cost 
and  improve  performance  (reliability).  The 
government  achieved  its  objective  in  each 
case.  However,  the  government  objectives 
and  the  resulting  level  of  success  will  likely 
vary  on  a  case  by  case  basis,  even  within  a 
given  program. 

The  ALCM,  GLCM,  and  SLCM  programs 
were  all  programs  of  the  highest  national 
importance  and  held  a  BRICK-BAT  DX 
priority  rating  early  in  the  FSB  phase.  The 
ALCM  program  had  the  earliest  deployment 
date  of  the  three,  and  this  coupled  with  the 
large  initial  production  quantity  (3418) 
necessitated  the  timely  fabrication  and 
delivery  of  the  sustainer  engine  and  INE. 

In  the  three  cruise  missile  cases,  the  first 
open  competition  year  was  two  to  three 
years  after  agreement  was  reached  to  hold 
the  competition.  Had  other  dual  sourcing 
strategies  been  used  (e.g.,  an  alternate 
design),  the  length  of  time  to  the  first 
competition  year  would  have  likely 
increased,  thus  diminishing  the  government's 


ability  to  achieve  its  primary  objective  in  the 
sustainer  engine  and  INE  cases  (meeting  the 
ALCM  delivery  schedule  and  decreasing  INE 
cost),  as  well  as  in  the  AUR  case  (reduce 
GLCM  and  SLCM  air  vehicle  production 
cost,  improve  reliability,  and  meet  the 
delivery  schedules). 

An  identical  design,  leader/follower  form  of 
dual  sourcing  was  used  in  each  case,  but  this 
may  not  be  optimal  or  even  desirable  for 
other  programs.  In  a  broader  sense,  the 
decision  as  to  whether  or  not  production 
phase  competition  is  warranted  varies  with  a 
host  of  economic,  programmatic,  and 
technical  considerations.  A  partial  list  of 
relevant  considerations  includes  the  rationale 
for  competition  (e.g.,  purely  economic), 
availability  of  an  acceptable  second  source, 
the  degree  of  potential  rivalry  between  the 
contractors,  economic  considerations  (e.g., 
annual  production  rate  and  total  quantity), 
production  quantity  constraints  per 
contractor  (e.g.,  a  guaranteed  minimum), 
government  source  selection  criteria,  and  the 
type  of  competition.  Few  simple  universal 
guidelines  apply  as  to  whether  or  not  a 
production  phase  competition  will  be 
successful. 

For  example,  the  government  must 
separately  view  and  analyze  the  potential 
competition  structure  versus  the  likely 
contractor  behavior.  These  considerations 
both  shaped  the  government  acquisition 
strategy  and  impacted  the  degree  of  success 
achieved  in  the  cruise  missile  engine  and  INE 
competitions. 

Both  WIC  (engine)  and  Litton  (INE)  claimed 
rights  to  data.  The  government  could  not 
delay  ALCM  deployment  to  argue  the  legal 
merits  of  these  contractor  claims.  In  all 
likelihood,  the  WIC  claims  were  weak  or 
negligible  by  the  time  ALCM  production 
began,  while  those  made  by  Litton  were 
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viable.  However,  even  in  the  engine  case  the 
government  had  little  real  choice  but  to 
accept  the  WIC  claim  because  of  the  tight 
ALCM  deployment  schedule.  In  effect,  there 
were  no  close  substitutes  for  the  WIC  FI 07 
engine.  Any  other  engine  would  have 
required  moderate  development,  and  was 
undesirable  from  a  cost,  performance,  and 
schedule  point  of  view.  Similarly,  in  the  INE 
case  the  best  available  substitute  was  not 
acceptable  from  a  cost  and  schedule 
perspective. 

Although  both  WIC  and  TCAE  had  already 
been  competitors  in  the  cruise  missile  engine 
business,  the  substantial  constraints  imposed 
on  the  competition  structure  resulting  from 
the  WIC  claim  to  data  rights  precluded 
TCAE  from  ever  winning  the  majority  of  a 
yearly  split-buy  competition.  On  the  other 
hand,  Litton  and  LSL  were  sister  divisions  of 
the  same  company,  and  this  might  by  itself 
suggest  a  weak  price  competition.  However, 
Litton  and  LSL  had  a  history  of  strong 
rivalry  when  competing  on  prior  programs. 
This  coupled  with  less  restrictive 
requirements  for  a  Litton  monthly  minimum 
award  quantity  and  a  LSL  maximum  monthly 
award  quantity  led  to  a  substantially  stronger 
price  competition  than  in  the  sustainer  engine 
case. 

The  AUR  production  competition  success 
was  largely  due  to  four  reasons.  First,  each 
contractor  strongly  desired  obtaining 
business  in  the  other's  arena.  Second,  the 
two  experienced  cruise  missile  contractors 
cross-licensed  their  respective  subsystems 
and  transferred  the  technology  to  each  other 
at  minimal  charge.  Third,  a  strong  rivalry 
existed  between  the  two  companies.  Fourth, 
much  of  the  credit  for  the  success  belongs  to 
JCMPO  personnel,  who  envisioned  the 
possible  competition  structure  and  worked 
through  many  substantial  issues  with  both 
contractors,  who  were  not  initially  receptive 


to  the  idea  of  production  phase  dual 
sourcing. 
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ABSTRACT 

The  "President's  Blue  Ribbon  Commission 
on  Defense  Management,"  in  1986, 
identified  three  military  programs,  the 
Minuteman  I,  Polaris,  and  the  Air  Launched 
Cruise  Missile  (ALCM),  as  being  examples 
of  defense  programs  with  streamlined 
procedures  that  achieved  accelerated 
(development)  schedules  typically 
associated  with  successful  commercial 
programs. 

The  Minuteman  I,  Polaris,  and  ALCM 
acquisition  histories  were  examined  to 
identify  common  characteristics  present  and 
to  determine  if  lessons  learned  can  readily 
be  transferred  to  other  DoD  programs  to 
potentially  improve  the  odds  of  program 
success. 

INTRODUCTION 

The  "President's  Blue  Ribbon  Commission 
on  Defense  Management,"  in  1986,  stated 
that:  "It  is  clear  that  major  savings  are 
possible  in  the  development  of  weapon 
systems  if  the  Department  of  Defense  (DoD) 
broadly  emulates  the  acquisition  procedures 
used  in  outstanding  commercial  programs" 
[1]. 


The  Commission  also  stated  that:  "It  is  clear 
from  our  earlier  description  that  defense 
acquisition  typically  differs  from  this 
commercial  model  in  almost  every  respect" 
[2].  However,  the  Commission  did  name 
three  military  programs,  the  Minuteman  I, 
Polaris,  and  (post-Milestone  II)  long-range 
Air  Launched  Cruise  Missile  (ALCM),  as 
being  examples  of  defense  programs  with 
streamlined  procedures  that  achieved 
accelerated  (development)  schedules 
typically  associated  with  successful 
commercial  programs  [3]. 

Only  the  post-Milestone  II  portion  of  the 
ALCM  program  used  streamlined 
acquisition  procedures.  The  long-range 
ALCM  program  (AGM-86B)  was 
effectively  initiated  as  a  result  of  the  cruise 
missile  Milestone  II  decision  memorandum 
on  January  14,  1977.  Prior  to  this  time  the 
long-range  ALCM  had  only  been  a  paper 
study.  Now  it  was  to  be  developed  and 
given  priority  over  the  existing  short-range 
ALCM  (AGM-86A)  the  Air  Force  had  been 
developing  [4].  (Shortly  thereafter,  the 
short-range  ALCM  program  ended.)  The 
ALCM  program  was  transferred  to  the  Joint 
Cruise  Missiles  Project  Office  (JCMPO)  as  a 
result  of  the  Milestone  II  decision 
memorandum.  (The  JCMPO  was  also 
created  as  a  result  of  the  Milestone  II 
decision  memorandum.)  However,  it  was 
not  until  President  Carter  canceled  the  B-1A 
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program  on  June  30,  1977  and  the 

September  30,  1977  memorandum  from 
Undersecretary  of  Defense  for  Research  and 
Engineering  (USDR&E)  Dr.  William  Perry 
to  the  Secretaries  of  the  Air  Force  and  Navy, 
that  the  JCMPO  was  given  sufficient 
external  support  in  order  to  implement  a 
streamlined  acquisition  program  for  the 
long-range  ALCM.  (The  ALCM  flyoff  was 
initiated  in  the  same  Dr.  Perry  memorandum 
eight  months  after  the  program  entered  Full 
Scale  Development  (FSD,  now  Engineering 
and  Manufacturing  Development  (EMD).) 

The  development  program  acquisition 
histories  of  the  Minuteman  I,  Polaris  A-l, 
and  ALCM  systems  were  examined  to 
identify:  (1)  some  common  characteristics 
of  these  programs  and  (2)  whether  or  not  the 
lessons  learned  can  be  transferred  to  other 
DoD  programs  to  potentially  improve  the 
odds  of  program  success. 

COMPARISON  OF  MINUTEMAN  I, 
POLARIS  A-l,  AND  ALCM 
DEVELOPMENT  PROGRAMS 

The  Minuteman  I  [5],  Polaris  A-l  [6],  and 
ALCM  [7]  program  schedules  are 
summarized  in  Figures  1,  2,  and  3, 
respectively. 

Priority  Rating 

Each  program  was  a  program  of  the  highest 
national  importance,  and  held  a  BRICK¬ 
BAT  DX  priority  rating.  This  rating 
provided  assurance  for  availability  of 
materials,  components,  and  other  resources 
(e.g.,  test  ranges)  in  the  event  of  conflict 
with  commercial  or  lesser  important  defense 
contracts.  The  Minuteman  I,  Polaris  (Fleet 
Ballistic  Missile  Program),  and  ALCM  were 
assigned  this  rating  in  September  1959, 
November  1955,  and  February  1978. 


Program  Management  Autonomy 

Each  program  had  considerable  management 
autonomy.  However,  this  did  not  prevent 
either  bureaucratic  encroachment  or 
considerable  programmatic  turbulence  from 
occurring  during  missile  development 
(Minuteman  I)  or  following  development 
completion  (Polaris  A-l  and  ALCM). 

Conflicts  between  programmatic  success 
and  bureaucratic  success  exist  and  generally 
increase  as  a  program  nears  completion.  A 
potential  cause  of  this  problem  is  that 
tendencies  towards  suboptimization  inherent 
in  project  management  are  exacerbated 
when  the  urgency  of  the  mission  declines 
[8], 

In  the  Minuteman  I  case,  Lt.  Gen.  Bernard 
Schriever  was  the  commander  of  the  Air 
Force  Ballistic  Missile  Division  and  the 
deputy  commander  of  the  Air  Research  and 
Development  Center  (ARDC,  which  later 
became  Air  Force  Systems  Command).  In 
essence,  Lt.  Gen.  Schriever  was  his  own 
supervisor,  which  gave  him  very  strong 
control  over  funding  and  review  authority 
through  1958  (early  FSD  program  phase). 
However,  upon  becoming  ARDC 
commander  in  March  1959,  Lt.  Gen. 
Schriever  abolished  special  management  by 
exception  for  the  Minuteman  I  program 
office  and  the  program  no  longer  possessed 
such  uncommon  reporting  and  protective 
mechanisms  [9]. 

Minuteman  I  management  independence 
had  declined  considerably  just  prior  to  the 
March  1960  production  decision,  as 
evidenced  in  an  excerpt  of  a  letter  from  Lt. 
Gen.  Schriever  to  Gen.  Thomas  White 
(Chief  of  Staff,  USAF):  ..."To  insure  a 
timely  and  effective  force,  our  weapon 
system  managers  must  possess  the  authority 
to  effect  necessary  program  changes.  The 
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continual  encroachment  upon  this  authority 
is  of  great  concern  to  me,  and  I  urge  that  you 
consider  a  return  to  the  streamlined 
management  principles  which  were  so 
effective  in  the  Thor  and  Atlas  programs." 
[10] 

By  February  1961,  the  Minuteman  I 
management  situation  had  become  so 
difficult  that  Lt.  Gen.  Schriever  expressed 
great  concern  to  Gen.  White.  Lt.  Gen. 
Schriever  felt  that  although  the  Minuteman 
program  was  a  high  risk  program  requiring 
unusual  measures  in  all  respects  to  protect 
the  operational  dates,  the  program  was  being 
managed  in  a  routine  fashion  [11]. 

Similarly,  MGen.  Osmund  J.  Ritland, 
Commander  of  the  Air  Force  Ballistic 
Missile  Division,  in  a  February  17,  1961 
letter  to  Lt.  Gen.  Schriever,  drew  attention  to 
how  the  Navy  had  been  supporting  the 
Polaris  program.  MGen.  Ritland  indicated 
that  the  Navy  had  rallied  to  support  Polaris 
development,  and  that  there  had  been  no 
change  in  program  management  since  the 
beginning  of  the  program.  The  program 
director  had  been  given  total  authority  and 
resources  to  carry  out  his  assignment.  For 
this:  “...he  had  been  asked  for  only  one 
thing— results”.  [12]. 

The  most  important  Minuteman  I 
management  strengthening  measures 
occurred  on  February  23,  1961.  Gen.  White 
informed  all  deputies,  directors,  and  chiefs 
of  comparable  offices  that  the  Minuteman 
undertaking  was  a  “...crash  program.” 
Therefore,  all  matters  pertaining  Minuteman 
high  level  program  review  were  given 
“overriding  priority”,  and  reviews  of  the 
program  were  to  be  limited  to  a  single 
review  in  the  Air  Staff  with  ARDC 
participation  [13]. 


In  the  Polaris  case,  the  Special  Projects 
Office  quickly  learned  that  a  reputation  for 
managerial  efficiency  made  it  difficult  for 
anyone  to  challenge  the  Polaris  development 
plans.  PERT  (Program  Evaluation  and 
Review  Technique,  a  computerized 
planning,  scheduling,  and  control  technique 
first  made  public  in  mid-1958),  the  Polaris 
Management  Center,  and  other  management 
innovations  produced  a  protective  veneer  to 
allow  Polaris  development  [14].  The  degree 
of  program  protection  provided  by  these 
innovations  was  of  equal  or  greater  value 
than  the  intrinsic  management  efficiency 
benefits  of  the  techniques.  They  allowed  the 
technical  staff  to  work  relatively  unhindered 
and  protected  them  from  concerned  but 
potentially  disruptive  outside  officials. 

Decentralization  and  competition  were  key 
components  of  the  Polaris  management 
strategy,  which  provided  nearly  self¬ 
regulating  control  over  the  Polaris 
development  and  its  developers.  Through 
decentralization,  authority  to  act  was  given 
to  those  closest  to  the  problems,  yet 
competition  among  the  program  office 
branches  and  contractors  assured  the  central 
staff  that  decisions  affecting  the  vital  needs 
of  the  entire  system  would  be  brought  to 
their  attention  [15].  However,  Department 
of  the  Navy  restructurings  in  1963  and  1966 
eliminated  much  of  the  Special  Project 
Office  independence  and  management  by 
exception  [16].  Although  this  did  not 
greatly  affect  the  Polaris  series  of  missiles 
(A-l,  A-2,  and  the  initial  A-3),  it  did  impact 
all  subsequent  Navy  ballistic  missile 
programs. 

An  Executive  Committee  (EXCOM)  was 
established  by  the  USDR&E  Dr.  William 
Perry,  in  September  1977  (early  in  FSD)  to 
provide  programmatic  and  fiscal  direction  to 
monitor  the  progress  of  the  ALCM  flyoff 
and  other  cruise  missile  variants.  The 
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EXCOM  was  not  a  voting  group;  rather  its 
purpose  was  to  review  and  discuss  in  an 
attempt  to  establish  a  consensus.  In  the 
absence  of  a  consensus,  Dr.  Perry  would  act 
as  required  and  report  dissenting  opinions  to 
the  Secretary  of  Defense  along  with 
recommendations  for  action.  Normal 
channels  remained  open  to  the  Services  to 
express  dissent.  Another  EXCOM  feature 
was  that  it  provided  a  forum  for  an 
expeditious  review  of  problem  areas.  In 
addition,  through  its  high-level  OSD  and 
Service  membership,  and  the  use  of  action 
item  assignments,  EXCOM  interaction  with 
the  JCMPO  could  potentially  minimize 
program  cost  and  schedule  risk  [17]. 

The  EXCOM  was  a  key  element  in  the 
JCMPO  management  approach.  In  practice, 
the  EXCOM  served  several  functions.  One 
function  was  to  provide  a  periodic  and 
structured  forum  for  examining  problem 
areas.  Another  function  was  that  the 
EXCOM  members  had  enough  authority  to 
resolve  problems  quickly,  including 
resolution  of  funding  shortages  [18]. 

The  ALCM  began  transitioning  back  to  the 
Air  Force  from  the  JCMPO  following  its 
Milestone  III  review  in  March  1980.  The 
cruise  missile  project  EXCOM  was 
discontinued  after  its  final  meeting  on 
January  8,  1981.  After  that,  the  JCMPO 
director  had  no  effective  formal  mechanism 
for  resolving  issues  between  the  Air  Force 
and  Navy. 

Although  the  cessation  of  the  EXCOM  did 
not  affect  ALCM  development,  it  did 
potentially  impact  development  of  the  Air 
Force  Ground  Launched  Cruise  Missile  and 
the  Medium  Range  Air-to-Surface  Missile 
programs.  Another  effect  was  a 
psychological  one  in  the  minds  of  some 
JCMPO  members  and  associated  service 
officials.  After  the  EXCOM  was 


discontinued,  some  personnel  took  this 
action  to  mean  in  part  that  the  level  of  Office 
of  the  Secretary  of  Defense  (OSD)  support 
for  the  cruise  missile  project  had 
diminished.  Although  it  was  impossible  to 
quantify,  this  factor  undoubtedly  had  some 
non  beneficial  impact  on  the  cruise  missile 
project  [19]. 

Early  Program  Support 

None  of  the  three  programs  had  substantial 
support  early  in  its  history,  but  did  receive 
substantial  support  by  the  middle 
development  stages. 

For  example,  the  Minuteman  I  and  Polaris 
programs  were  initiated  after  the  four 
"approved"  ballistic  missile  programs  (Atlas 
Intercontinental  Ballistic  Missile  (ICBM), 
Titan  ICBM  (backup),  Thor  Intermediate 
Range  Ballistic  Missile  (IRBM),  and  Jupiter 
IRBM)  were  identified  in  September  1955 
by  President  Eisenhower  and  the  DoD  [20]. 
In  addition,  early  attempts  to  accelerate  the 
Minuteman  program  schedule  were  not  well 
received.  Just  prior  to  the  Minuteman  I FSD 
start,  and  11  months  after  the  first  Soviet 
Sputnik  was  launched,  W.  M.  Holaday, 
Department  of  Defense  Director  of  Guided 
Missiles,  wrote  to  James  H.  Douglas, 
Secretary  of  the  Air  Force,  on  September  17, 
1958,  noting  that  the  Minuteman  program: 
“is  not  in  consonance  with  my  desire  for  an 
orderly  step-wise  development  program. 
The  plan  submitted  is  characterized  by  the 
compressed  development  schedules 
associated  with  the  so-called  crash  programs 
such  as  Atlas  and  Titan  which,  while 
justified  by  the  urgency  of  the  requirement 
for  an  early  ICBM  capability,  are  not 
conducive  to  maximizing  operational 
effectiveness  or  minimizing  costs.”  [21] 

Minuteman  I  had  a  less  hectic  development 
program  than  the  Polaris  A-l,  partly  because 
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it  was  in  the  awkward  position  of  being 
competitive  with  more  advanced  liquid-fuel 
ballistic  missiles  being  developed  by  the  Air 
Force.  Essentially  all  the  critical 
uncertainties  of  Minuteman  technology  were 
reasonably  well  in  hand  by  1957,  but 
activating  a  full-scale  development  program 
would  inevitably  cause  a  diversion  of  effort 
from  the  other  ballistic  missile  programs  to 
which  the  Air  Force  had  commitments. 
Before  Sputnik  cut  the  purse  strings, 
Minuteman  I  could  have  been  developed 
only  at  the  price  of  limiting  expenditures  of 
the  Atlas  or  Titan  programs  [22].  The  effect 
of  the  Sputnik  furor  in  late  1957  and  of  the 
political  squabbling  that  sputtered  through 
the  next  three  years  was,  in  part,  the 
accelerated  development  of  the  Minuteman  I 
program  (which  was  initiated  on  May  20, 
1959). 

Similarly,  by  March  1957  the  Navy’s 
Special  Projects  Office  had  settled  on  the 
general  specifications  of  the  Polaris  missile, 
submarine,  undersurface  launch  system,  and 
related  components.  Acceleration  of  the 
original  program  on  December  9,  1957  and 
substantial  funding  authorizations  followed 
Sputnik.  Cutting  rather  more  than  two  years 
from  the  earlier  schedules  was  achieved  by 
compressing  schedules,  eliminating  test 
sequences,  and  by  relaxing  both  the  range 
and  accuracy  specifications  [23]. 

The  long-range  ALCM  faced  initial 
opposition  from  parts  of  the  Air  Force  in 
favor  of  its  much  higher  priority  B-1A 
coupled  with  the  short-range  ALCM  prior  to 
President  Carter's  cancellation  of  the  B-1A 
on  June  30,  1977  [24]. 

In  addition,  none  of  the  programs  had 
considerable  opposition  from  the  scientific 
community  beyond  the  early  development 
stage. 


Program  Funding  and  Funding 
Turbulence 

While  each  program  received  the  necessary 
funding,  potential  funding  shortfalls 
occurred  which  produced  program 
turbulence  in  the  short  run. 

In  the  Minuteman  I  case,  an  accelerated 
development  plan  was  approved  on  May  20, 

1959  (mid-way  through  FSD).  By 
November  1959,  the  Minuteman  I 
contractors  were  reporting  the  potential  for 
large  cost  increases.  The  cost  increases  were 
the  result  of  externally  directed  changes  to 
the  Minuteman  program,  including  the 
accelerated  schedule.  Initially,  the 
corresponding  increases  in  the  program 
budget  did  not  match  these  externally 
directed  changes,  and  substantial  increases 
in  Minuteman  I  funding  were  necessary 
during  the  early  portion  of  the  missile 
production  phase  in  FY60-FY62  [25]. 

In  the  Polaris  case:  "An  unexpected 

technical  crisis  would  force  internal 
reprogramming  of  funds  to  be  undertaken. 
But  inevitably  money  for  all  important 
activities  was  found,  even  if  the  lost  time 
could  not  always  be  recovered.  Program 
officials  learned  quickly  to  request  generous 
contingency  appropriations."  [26] 

On  December  9,  1957  (at  the  start  of  FSD), 
the  Secretary  of  Defense  authorized 
acceleration  of  the  Polaris  program  to 
deploy  the  first  Polaris  weapon  system  in 

1960  [27].  Shortly  thereafter,  on  February 
12,  1958,  President  Eisenhower  signed  the 
FY58  Supplemental  Appropriation  Act, 
including  funds  for  construction  of  the  first 
three  Polaris  submarines.  Construction  had 
begun  in  January  1958  using  funds 
“borrowed”  from  other  Navy  programs  [28]. 
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In  the  ALCM  case,  the  FSD  program  flyoff 
completion  date  slipped  approximately  three 
months  during  the  course  of  the  competition 
(from  November  1979  to  February  1980) 
because  of  the  late  receipt  of  the  FY78 
supplemental  appropriation  [29],  which  was 
dedicated  to  the  ALCM  program. 

While  it  is  difficult  to  accurately  estimate 
the  impact  of  program  acceleration  on  the 
development  cost,  the  resulting  increases 
were  not  minor  for  Minuteman  I  and 
ALCM.  In  the  Minuteman  I  case,  it  is 
estimated  that  program  acceleration  added 
roughly  45  percent  to  the  development  phase 
cost,  and  was  used  for  overtime  for 
contractor  employees  and  dual  sources  for 
critical  items  and  tests  [30].  In  the  ALCM 
case,  a  41  percent  cost  growth  occurred  in 
the  development  phase-the  largest 
contributors  to  this  growth  were  the 
accelerated  FSD  schedule  and  some 
additional  requirements  imposed  during  the 
flyoff  [31], 

Necessary  Technology  Advancement 

Each  program  represented  a  moderate 
technology  advance  required  across  the 
entire  system,  with  a  considerable  advance 
in  the  state  of  the  art  required  for  only  a  few 
subsystems.  In  addition,  each  program  was 
the  beneficiary  of  important  advances  made 
from  prior  technology,  development,  or 
operational  programs. 

For  example,  in  the  Minuteman  I  and  Polaris 
A-l  programs,  important  advances  had 
already  been  made  in  the  Atlas,  Thor,  and 
Titan  ballistic  missile  programs,  including 
reentry  physics,  development  of  electronic 
controls,  and  inertial  guidance.  In  addition, 
key  solid  propulsion  research  had  been 
started  in  the  mid  1950s  that  provided  an 
excellent  bridge  to  Minuteman  I  and  Polaris 
A-l  requirements.  The  "breakthrough"  for 


each  came  in  1955  (prior  to  the 
commencement  of  either  program)  with 
demonstrations  that  large-grain,  double-base 
solid  propellants  could  be  reliably  ignited 
and  burned  [32]. 

In  the  ALCM  case,  key  turbofan  engine 
development  had  begun  almost  five  years 
prior  to  the  Milestone  II  decision  to  initiate 
development  of  the  long-range  ALCM  in 
January  1977  [33],  Likewise,  the  guidance 
system  Inertial  Navigation  Element  (INE) 
was  closely  related  to  a  unit  that  had  already 
been  developed  and  tested  for  the  F-15  and 
other  military  aircraft.  In  effect,  the 
technical  characteristics  of  both  the  ALCM 
engine  and  INE  were  well  in  hand  prior  to 
the  cruise  missile  Milestone  II  decision  on 
January  14,  1977. 

Common  areas  of  technology  advancement 
used  by  all  three  programs  included 
improved  propulsion  and  warheads. 

Some  common  key  technology  areas  for  the 
Minuteman  I  and  Polaris  A-l  that  required 
moderate  development  included:  (1)  high 
energy,  solid  propellants  with  advanced 
binders  and  additives,  and  uniform 
character,  which  provided  improved  thrust 
and  reliability;  (2)  improved  materials  (e.g., 
with  increased  strength  and  reduced  weight), 
which  contributed  to  increased  range;  (3) 
thrust  vector  control  and  thrust  termination 
devices,  which  improved  missile  accuracy; 
(4)  ablative  reentry  vehicle  development, 
which  reduced  payload  weight  and  reentry 
dispersion,  and  increased  accuracy;  and  (5) 
reduced  thermonuclear  warhead  weight, 
which  reduced  payload  weight,  thus 
increasing  range,  for  a  given  warhead  yield. 

ALCM  key  technology  areas  included:  (1)  a 
low  specific  fuel  consumption,  high  thrust, 
small  volume  turbofan  engine  with  high 
altitude  startup,  which  increased  range;  (2)  a 
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special  warhead  design;  (3)  a  low 
observables  air  vehicle,  using  a  special 
airframe,  missile  radar  altimeter,  and  engine 
design,  which  increased  survivability;  and 
(4)  terrain  contour  matching  and  terrain 
following  software,  which  increased  missile 
accuracy  and  survivability. 

Missile  Thrust/Weight  Ratio 

The  initial  thrust/weight  ratio  associated 
with  engine  propulsion,  coupled  with  missile 
weight,  proved  to  be  optimistic  for  each 
program.  This  led  to  a  decrease  in  range  for 
the  first  wing  of  Minuteman  I  deployed  and 
the  Polaris  A-l,  and  is  indicative  of  an  initial 
set  of  system  requirements  that  exceeded  the 
feasible  level  of  performance  that  could  be 
achieved  for  the  estimated  level  of  cost  and 
schedule. 

Sacrifices  versus  desired  performance  levels 
were  made  for  each  program  in  order  to 
meet  the  initial  deployment  schedule.  For 
the  Minuteman  I  and  Polaris  A-l,  a 
reduction  in  performance  occurred  along 
with  an  increase  in  development  phase  cost 
in  the  Minuteman  I.  Approximately  60 
percent  of  the  range  reduction  in  the  first 
Minuteman  I  wing  was  corrected  in  the 
second  Minuteman  I  wing  deployed  in  1963, 
and  virtually  all  of  the  Polaris  A-l  missile 
range  reduction  was  removed  in  the  Polaris 
A-2  missile.  The  range  (performance) 
reduction  was  accepted  in  the  first 
Minuteman  I  and  Polaris  A-l  missiles 
deployed  in  order  to  reduce  program 
schedule  risk.  In  the  ALCM  case,  no  early 
attempt  was  made  to  correct  the  problem 
since  the  demonstrated  performance  level 
was  adequate.  The  government  emphasis  in 
the  ALCM  program  was  to  ensure  the 
eventual  production  delivery  schedule  while 
maintaining  development  phase  cost  and 
performance. 


Use  of  Parallel  R&D 

Parallel  research  and  development  (R&D) 
and  simultaneous  exploration  of  several 
alternatives  were  used  extensively  as  risk 
reduction  measures  for  selected  key 
technology  areas  that  required  considerable 
advances  in  the  state  of  the  art.  Parallel 
R&D  had  been  recognized  by  this  time  as  a 
key  tool  for  reducing  program  risk  when 
technology  advances  were  required, 
particularly  on  a  short  schedule  [34], 

The  extended  parallel  R&D  present  in  the 
Minuteman  I,  Polaris  A-l,  and  ALCM 
programs  greatly  reduced  the  resulting  risk 
associated  with  the  relatively  short  program 
schedule  length,  coupled  with  the 
technology  development  that  was  necessary. 

For  example,  the  Minuteman  I  program  had 
as  many  as  three  different  contractors 
developing  missile  propulsion  stages  during 
1956-1961  [35].  The  parallel  R&D  efforts 
were  maintained  until,  and  in  the  case  of  the 
third  stage,  after  the  program  production 
decision  in  March  1960. 

For  Polaris:  "In  the  launch  area,  1 1  different 
methods  of  ejecting  a  missile  from  a 
submerged  submarine  were  said  to  have 
been  simultaneously  considered.  Similarly, 
in  the  navigation  area  at  least  two  teams 
approached  the  problem  of  developing  an 
inertial  navigation  system  and  several 
substitute  navigation  schemes  were  also 
explored."  [36] 

In  the  ALCM  program,  the  engine  and  INE 
were  well  advanced  in  development  prior  to 
initiating  the  long-range  ALCM  program  in 
January  1977.  Each  item  had  undergone 
parallel  development  with  two  contractors 
culminating  with  a  selected  design  prior  to 
beginning  FSD.  For  example,  the  engine 
and  INE  downselect  to  a  single  development 
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contractor  occurred  in  April  1973  and 
October  1975,  respectively.  However,  the 
required  ALCM  deployment  schedule 
necessitated  increasing  the  production  rate 
for  the  engine  and  INE.  An  identical  design 
dual  source  production  competition  strategy 
was  established  during  FSD  for  the  engine 
and  INE  in  August  1978  and  October  1978, 
respectively,  to  meet  the  ALCM  deployment 
schedule  [37]. 

Development  Phase  Flight  Tests 

Although  the  Minuteman  I,  Polaris  A-l,  and 
ALCM  programs  had  fast  paced  schedules, 
their  early  flight  test  programs  were  not 
particularly  successful,  and  deployment 
occurred  before  the  systems  were  fully 
mature.  (Each  system,  though,  still 
possessed  moderate-to-high  operational 
utility.) 

Four  of  the  first  10  Minuteman  I  flight  tests 
(beginning  February  1,  1961)  were  failures 
or  partial  failures.  (In  addition,  the  first 
Minuteman  I  flight  was  roughly  1 1  months 
after  the  initial  production  decision!)  Flights 
of  the  first  five  Polaris-configured  flight  test 
vehicles  (AX  series,  beginning  September 
24,  1958)  ended  in  failures.  Both  the 
candidate  Boeing  and  General  Dynamics 
long-range  ALCMs  suffered  four  early 
terminations  each  in  their  10  missile  FSD 
flyoff  (beginning  August  3,  1979  and  July 
17,  1979,  respectively). 

Consequently,  four  or  more  of  the  first  10 
flight  tests  were  failures  or  partial  failures 
for  each  program.  In  effect,  the  program 
managers  and  cognizant  DoD  personnel  had 
considerable  confidence  in  each  weapon 
system,  and  protected  them  from  adverse 
external  reactions.  Any  DoD  program  today 
having  a  success  factor  of  <  60  percent  in  its 
first  10  flight  tests,  regardless  of  its  flight 
test  record  afterwards,  would  almost 


certainly  encounter  considerable  monitoring 
from  external  organizations  (e.g.,  the 
Congress),  that  might  adversely  impact 
program  cost,  performance,  and/or  schedule, 
if  not  program  termination. 

Summary  of  Some  Common  Program 
Characteristics 

At  this  point  I  will  summarize  some  of  the 
key  characteristics  of  the  Minuteman  I, 
Polaris  A-l,  and  ALCM  programs  discussed 
so  far. 

First,  each  program  was  a  program  of  the 
highest  national  importance  and  held  a 
BRICK-BAT  DX  priority  rating.  Second, 
each  program  was  not  wholly  insulated  from 
externally  induced  program  impacts  during 
the  development  phase.  Similarly,  the 
programs  were  impacted  by  service  and 
DoD-level  organizational  and  policy 
changes.  Third,  each  program  had  only 
modest  to  moderate  support  until  mid-way 
in  the  development  phase.  To  a  great  degree 
factors  external  to  the  Minuteman  I,  Polaris, 
and  ALCM  program  offices  led  to  increased 
program  support.  Fourth,  while  each 
program  received  the  necessary  funding, 
potential  funding  shortfalls  occurred  that 
produced  moderate  program  turbulence  in 
the  short  run.  In  addition,  the  accelerated 
program  schedules  led  to  moderate 
development  phase  cost  growth  in  the 
Minuteman  I  and  ALCM  programs.  Fifth, 
each  program  required  a  moderate 
technology  advance  across  the  entire  system, 
with  a  considerable  advance  in  the  state  of 
the  art  required  for  only  a  few  subsystems. 
Each  program  was  the  beneficiary  of 
important  advances  made  from  prior 
technology  or  operational  programs.  Sixth, 
the  initial  thrust/weight  ratio  associated  with 
engine  propulsion,  coupled  with  missile 
weight,  proved  to  be  optimistic.  Seventh, 
parallel  R&D  and  simultaneous  exploration 
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of  several  alternatives  were  used  extensively 
used  as  risk  reduction  measures  for  a 
number  of  the  key  technology  areas.  Eighth, 
the  development  phase  flight  test  programs 
were  not  particularly  successful.  In  today's 
acquisition  environment,  this  would  likely 
lead  to  program  schedule  slippage,  if  not 
termination. 

CAN  THE  LESSONS  LEARNED  BE 
TRANSFERRED  TO  OTHER  DoD 
PROGRAMS?  (CONCLUSIONS) 

Given  the  common  characteristics  of  the 
three  programs,  I  will  now  briefly  discuss 
whether  or  not  lessons  learned  from  the 
Minuteman  I,  Polaris  A-l,  and  ALCM 
programs  can  be  transferred  to  other  DoD 
programs. 

First,  the  success  of  the  Minuteman  I, 
Polaris  A-l,  and  ALCM  programs  was 
largely  due  to  an  unusual  convergence  of 
technological  opportunities  and  a  consensus 
on  national  needs. 

The  ballistic  and  cruise  missile  research 
programs  had  already  made  considerable 
strides  in  the  technology  associated  with  key 
subsystems  for  the  Minuteman  I,  Polaris  A- 
1,  and  ALCM  prior  to  the  FSD  (equivalent) 
program  phase.  Hence,  the  underlying 
technologies  needed  for  these  missiles  were 
already  on  the  necessary  path  to  yield  viable 
subsystems. 

In  addition,  there  was  a  clear  consensus  on 
national  needs  associated  with  the  perceived 
Soviet  ballistic  missile  gap,  coupled  with  the 
Sputnik  program,  that  assisted  both  the 
Minuteman  I  and  Polaris  A-l  beginning  in 
late  1957  [38], 

As  noted  by  Robert  Perry,  the  Minuteman  I 
and  Polaris  A-l  acceleration  decisions  were: 
“made  feasible  by  the  pace  of  technology— 


which  certainly  had  been  more  rapid  for 
ballistic  missiles  than  for  weapons 
contemporary  with  them.  But  the  decisions 
probably  would  not  have  been  made  as  they 
were  if  the  Soviet  Union  had  not  provided 
first  rate  motivation:  an  unmistakable 

threat.”  [39] 

Similarly,  upgraded  Soviet  air  defenses 
made  the  role  of  a  long-range  ALCM 
relatively  more  important  than  a  short-range 
ALCM  coupled  with  the  B-1A  penetrating 
bomber,  that  had  less  extensive  stealth 
characteristics  than  either  ALCM. 

The  breakthrough  that  permitted  the  rapid, 
extensive  deployment  of  these  missiles  was 
far  more  political  than  technological.  The 
political  consensus  permitted  virtually 
unrestricted  program  funding,  although 
moderate  short  run  funding  turbulence  did 
exist,  and  diminished  the  influence  of 
program  critics. 

In  effect,  the  convergence  of  technological 
opportunity  coupled  with  a  consensus  of 
national  needs  rarely  has  existed  to  the 
extent  that  it  did  for  the  Minuteman  I, 
Polaris,  and  ALCM  programs  except  for  the 
Manhattan  project,  the  Apollo  project,  and 
perhaps  a  few  others.  These  were  both 
fortunate  and  special  programs  in  a  number 
of  ways,  and  although  the  technological 
advances  made  in  each  program  were 
noteworthy,  the  political  environment, 
defined  through  a  consensus  of  national 
needs,  that  existed  in  each  case  played  a 
dominant  role  in  the  development, 
deployment,  and  success  of  these  systems. 
Without  both  technological  opportunities 
and  a  consensus  on  national  needs  it  is 
unlikely  that  many  military  programs  run 
with  a  conventional  acquisition  strategy  can 
achieve  the  accelerated  (development) 
schedules  typically  associated  with 
successful  commercial  programs. 
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In  some  cases  a  consensus  of  service  needs 
may  be  sufficient,  while  for  other  programs, 
a  consensus  of  DoD,  government  (including 
the  Congress),  or  even  national  needs  may 
be  necessary  to  insure  a  successful  program 
development  and  deployment  in  terms  of  the 
required  performance  and/or  schedule. 

As  Harvey  Sapolsky  stated  in  1972:  "Most 
government  undertakings,  whether  civilian 
or  military,  are  apparently  not  the 
beneficiaries  of  a  convergence  between 
technological  opportunities  and  political 
consensus."  [40] 

Second,  of  the  eight  common  Minuteman  I, 
Polaris  A-l,  and  ALCM  program 
characteristics  previously  identified  only 
two  are  both  desirable  and  somewhat  within 
the  control  of  the  government  program 
office  director.  These  include  the  need  for 
only  a  moderate  average  advance  in  the 
required  technology  state  of  the  art  and  the 
use  of  parallel  R&D  for  risk  reduction 
(assuming  adequate  funding  is  available). 

The  other  six  items  previously  mentioned 
either  require  actions  external  to  the 
government  program  office  to  implement 
(e.g.,  receiving  a  BRICK-BAT  DX  priority 
rating  and  service,  DoD-level  organizational 
and  policy  changes,  and  receiving  necessary 
funding)  or  represent  actions  with 
potentially  undesirable  effects  to  the 
program  (e.g.,  limited  early  support, 
optimistic  missile  thrust/weight  ratio,  and  a 
not  particularly  successful  development 
phase  flight  test  program.) 

Consequently,  applying  lessons  learned 
from  the  Minuteman  I,  Polaris  A-l,  and 
ALCM  programs  to  a  new  military 
development  program  may  assist  the 
program  to  some  extent  (e.g.,  parallel  R&D 
for  risk  reduction  if  funding  is  available). 
However,  there  is  no  guarantee  that  the 


program  will  achieve  the  same  degree  of 
success  as  in  the  Minuteman  I,  Polaris  A-l, 
and  ALCM  programs.  The  level  of  success 
achieved  by  these  programs  was  to  a  great 
extent  determined  by  both  necessary 
technological  opportunities  and  a  national 
consensus  of  needs  which  where  external  to 
the  government  program  offices,  and  even 
the  DoD  itself. 
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Minuteman  I  Schedule 


Polaris  Schedule 

Figure  2 
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ALCM  Development  History 

Figure  3 
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ABSTRACT 

Over  the  past  three  decades,  several 
approaches  have  been  tried  in  an  attempt  to 
improve  the  predictability  and  control  of 
military  systems  acquisition.  Yet  costs  grow 
and  schedules  slip,  while  the  final  product 
usually  has  performance  close  to  that 
specified. 

The  specific  sources  of  cost  and  schedule 
growth  may  vary  from  project  to  project,  but 
they  are  generally  associated  with  the 
existence  of  different  government  program 
office  and  contractor  objectives. 

A  microeconomic  analytical  framework  was 
created  to  model  the  military  systems 
acquisition  process.  The  hypothesized 
model  behavior  is  consistent  with  the 
statistical  results  obtained  from  analyzing 
the  cost,  performance,  and  schedule 
outcomes  of  a  broad  sample  of  military 
programs  spanning  the  1950s  through  1980s. 
The  same  preferences,  interactions  and 
outcomes  also  apply  to  highly  classified 
("Black")  programs  and  NASA  space 
programs. 

INTRODUCTION 

The  management  of  a  major  military  system 
acquisition  project  is  an  exceptionally 
challenging  task.  Given  its  complexity,  the 


large  amounts  of  money  frequently 
involved,  and  the  political  and  military 
consequences  riding  on  the  outcomes,  it  is 
understandable  that  considerable  attention 
and  effort  have  been  devoted  to  improving 
the  methods  of  military  systems  acquisition 
management.  Over  the  past  three  decades, 
several  different  acquisition  strategies  and 
numerous  detailed  management  approaches 
have  been  tried  in  an  attempt  to  improve  the 
predictability  and  control  of  project 
outcomes.  Improvements  have  been  sought, 
for  example,  through  changes  in  the  types  of 
contracts,  the  type  and  extent  of 
competition,  the  amount  of  management 
reporting  and  review,  and  the  distribution  of 
responsibility  between  the  service  project 
office  and  industrial  contractor.  Yet  the  goal 
of  a  predictable  and  controllable  project  has 
proved  elusive. 

Persistent  and  widespread  cost  and  schedule 
growth  in  U.  S.  military  systems  has  been 
documented  since  the  1950s;  in  contrast, 
system  performance  is  typically  very  close 
to  desired  levels.  However,  no  satisfactory 
theoretical  explanation  has  been  advanced  to 
date  (1994)  to  explain  this  behavior. 

While  the  specific  sources  of  cost, 
performance,  and  schedule  (C,P,S)  difficulty 
may  vary  from  one  project  to  the  next,  they 
are  generally  associated  with  different 
preferences  held  by  the  government  program 
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office  and  contractors,  coupled  with  a  poor 
understanding  of  the  range  of  possible 
outcomes,  which  themselves  are  conditioned 
by  program  structure.  A  microeconomic 
framework  is  presented  in  this  paper  that 
addresses  these  issues  and  models  the 
military  acquisition  process. 

BACKGROUND 

Marshall  and  Meckling  were  perhaps  the 
first  to  identify  cost,  performance,  and 
schedule  as  important  variables  for 
measuring  program  success  [1].  They 
evaluated  a  sample  of  aircraft  and  missile 
programs  with  equivalent  Engineering  and 
Manufacturing  Development  (EMD)  phase 
start  dates  in  the  1940s  and  1950s.  They 
found  the  average  ratio  of  the  most  recent 
and  earliest  production  cost  estimates  to  be 
between  2.4  and  3.2.  (A  ratio  (change 
value)  >  1  indicates  cost  or  schedule  growth 
or  performance  degradation.)  This 
corresponds  to  a  140  to  220  percent  increase 
in  cost.  They  also  found  the  average 
schedule  change  to  be  1.5  between  early 
estimates  of  first  operational  dates  and  the 
actual  first  operational  dates. 

Commenting  on  the  cost  growth  and 
schedule  slippage,  Marshall  and  Meckling 
said:  "Availability  (schedule)  predictions, 
like  cost  predictions  exhibit  both  a  decided 
bias  toward  overoptimism  and  substantial 
variation  in  the  extent  of  the  optimism."  [2] 

Perry,  et.  al.  estimated  cost,  performance, 
and  schedule  change  ratios  for  a  sample  of 
military  programs  with  start  dates  in  the 
1950s  and  1960s  [3],  They  found  the 
average  cost,  performance,  and  schedule 
change  to  be  1.44,  1.05,  and  1.15, 

respectively. 

Dews,  et.  al.  estimated  these  same  ratios  for 
a  sample  of  military  programs  with  start 


dates  in  the  1970s  [4],  They  found  the 
average  cost,  performance,  and  schedule 
change  to  be  1.34,  1.00,  and  1.13, 

respectively. 

Cost  and  schedule  growth  occurred  in  each 
of  these  program  samples,  but  where 
performance  results  were  evaluated, 
virtually  no  change  was  observed. 

Several  insightful  observations  exist  in  the 
literature  as  to  the  causes  of  military  system 
cost  and  schedule  growth. 

One  likely  cause  was  first  identified  by 
Marshall  and  Meckling  in  1959:  "Typically, 
in  weapons  development  great  emphasis  is 
placed  on  performance.  Most  new  weapons 
are  developed  around  specific  detailed 
performance  requirements  laid  down  by  the 
military  requirements  that  are  taken  very 
seriously.  The  penalties  incurred  by  the 
contractors  for  not  meeting  performance 
requirements  are  more  severe  than  for  failure 
to  meet  availability  schedules  or  failure  to 
live  within  original  cost  estimates.  As  a 
result,  whenever  circumstances  dictate  a 
retreat  from  early  plans,  it  is  usually  the 
costs  and/or  availability  that  gives  ground." 
[5] 

The  government  and  contractors  typically 
face  only  weak  disincentives  for  developing 
unrealistic  estimates  of  program  cost  and 
schedule,  as  identified  by  Marshall  and 
Meckling:  "Contractors  are  anxious  to  have 
their  proposals  accepted  by  the  military,  and 
the  military  itself  is  anxious  to  have 
development  proposals  supported  by  the 
Department  of  Defense  and  Congress.  The 
incentive  to  make  optimistic  estimates  is 
thus  very  strong.  On  the  other  hand,  the 
contractual  penalties  for  having  been 
overoptimistic  are  generally  small."  [6] 
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The  acquisition  process  of  U.  S.  military 
systems  has  been  distorted  in  this  way  for 
many  years.  Charles  J.  Hitch  and  Roland  N. 
McKean  stated  it  in  the  following  manner  in 
1960:  "Excessive  optimism  in  drawing  up 
performance  specifications  can  make  the 
development  so  difficult  that  it  must  fail,  or 
take  much  longer  and  cost  much  more  than 
planned,  or  require  a  downgrading  of  the 
requirements.  It  is  not  unusual  for  weapon 
system  requirements  to  be  so  optimistic  that 
several  inventions  or  advances  in  the  state  of 
the  art  are  needed  on  schedule  if  the 
development  is  to  succeed."  [7] 

The  Government  Accounting  Office  (GAO) 
recently  identified  several  issues  that  may 
lead  to  problems  in  major  weapons 
acquisition,  including:  overly  optimistic 
cost  and  schedule  estimates  leading  to 
program  instability  and  cost  increases; 
programs  that  cannot  be  executed  as  planned 
with  available  funds;  and  programs  being 
oversold  to  survive  [8]. 

The  GAO  also  attempted  to  identify  possible 
causes  for  these  problems,  and  stated: 
"While  there  are  many  reasons  for  these 
types  of  problems,  the  underlying  cause  of 
persistent  and  fundamental  problems  in 
DoD's  weapons  acquisition  process  is  a 
prevailing  culture  that  is  dependent  on 
generating  and  supporting  new  weapons 
acquisitions."  [9] 

In  summary,  distortions  in  the  military 
systems  acquisition  process  that  can  lead  to 
considerable  cost  and  schedule  growth  have 
been  noted  for  the  past  35  years  and 
validated  in  part  by  data  going  back  to  the 
1940s  and  1950s. 

Each  of  the  above  studies  has  contributed 
knowledge  about  military  program  C,P,S 
changes  and  some  of  their  underlying 
causes.  However,  a  void  exists  in  the 


literature  relative  to  a  mechanism  to  tie  cost, 
performance,  and  schedule  trends  together. 

This  paper  presents  a  comprehensive 
microeconomic  analytical  framework  that 
models  the  military  systems  acquisition 
process.  It  includes  government  and 
contractor  preferences;  external, 
programmatic,  and  technical  constraints; 
factors  that  influence  the  program  starting 
point;  dynamics  that  occur  during  the  course 
of  a  program;  and  likely  outcomes  predicted 
by  the  theoretical  framework. 

MICROECONOMIC  FRAMEWORK 

In  acquiring  U.  S.  military  systems,  the 
government  and  contractor  each  have  a  set 
of  objectives  regarding  the  projected  C,P,S 
outcomes.  The  government  generally 
prefers  lower  over  higher  cost  and  schedule, 
and  higher  performance,  while  the  contractor 
prefers  higher  cost,  performance,  and 
schedule. 

Lower  costs  are  desirable  to  the  government 
to  develop  more  military  systems  for  a  fixed 
budget  (or  the  same  number  for  a  reduced 
expenditure).  Shorter  schedules  that  enable 
the  system  to  become  operational  earlier 
enhance  the  force  structure  and  military 
balance  of  power.  Higher  performance 
permits  increased  operational  capability  for 
the  mission. 

Contractors  prefer  higher  costs  because  they 
increase  profits.  Longer  schedules  are  also 
desirable  to  maintain  a  stable  work  force  and 
a  long-term  working  relationship  with  the 
government,  which  gives  the  contractor  a 
potential  competitive  advantage  for  follow- 
on  or  future  contracts.  Contractors  prefer 
high  performance  to  improve  their  potential 
competitive  advantage  in  the  high 
technology  arena  that  often  provides  a 
potential  competitive  advantage. 
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The  production  schedule  is  generally  set  by 
high  level  government  decision  makers  (e.g., 
the  services  or  Congress),  based  upon  inputs 
from  the  project  office  and  contractors  (e.g., 
cost  versus  lot  quantity).  Hence,  the 
production  schedule  can  generally  be 
characterized  as  a  constraint  externally 
imposed  by  higher  level  government 
personnel. 

Given  the  government  and  contractor 
preferences,  the  next  step  is  to  consider 
programmatic  and  technical  constraints 
associated  with  cost,  performance,  and 
schedule. 

The  government  will  typically  have  in  mind 
a  maximum  program  cost  and  schedule 
length,  along  with  minimum  performance. 
The  contractor  will  often  have  its  own 
minimum  cost  and  schedule  length. 

A  technical  possibility  surface  encompasses 
the  region  of  feasible  C,P,S  solutions,  and  is 
the  technical  program  constraint.  Points  on 
the  technical  possibility  surface  or  any  two- 
dimensional  slice  of  the  surface  indicate 
efficient  (but  not  necessarily  optimal) 
solutions.  (This  is  given  by  points  B  and  C 
on  the  C:P  possibility  curve  in  Figure  1.) 
Points  lying  above  a  possibility  curve 
indicate  feasible,  but  inefficient  solutions. 
Those  points  below  a  possibility  curve 
indicate  an  infeasible  solution.  In  Figure  1 
for  the  C:P  case  (for  a  given  schedule),  point 
D  above  the  curve  is  an  inefficient 
combination  of  cost  and  performance,  since 
the  system  could  be  developed  at  the  same 
cost  but  with  higher  performance  by  moving 
to  the  right  or  at  the  same  level  of 
performance  with  less  cost  by  moving  down. 

A  point  to  the  right  of  the  C:P  curve  (A  in 
Figure  1)  is  infeasible  for  a  given  set  of 
input  constraints  (e.g.,  manufacturing 
processes,  technology  level,  program 


structure)  for  a  given  schedule.  This  point 
will  only  become  feasible  with  a  shift  to  the 
right  of  the  entire  C:P  curve,  thus  requiring, 
for  example,  improved  yield  for  a  given 
manufacturing  process  if  cost  is  held 
constant.  In  effect,  solution  point  A 
corresponds  to  an  inappropriate  schedule 
length  selected  for  the  specified  cost  and 
performance  levels. 

It  is  anticipated  that  the  last  few  percent  of 
the  maximum  performance  possible  will 
lead  to  increasingly  greater  program  cost 
and  schedule  length  (the  first  and  second 
derivatives  of  cost  with  respect  to 
performance  are  positive).  This  causes  a 
major  problem  in  military  system 
acquisition  where  performance  near  or 
exceeding  the  current  state  of  the  art  is 
typically  specified  for  new  development 
programs,  yet  insufficient  cost  and/or 
schedule  is  typically  budgeted  to  achieve  the 
desired  level  of  performance. 

The  government  and  contractor  preferences 
noted  earlier  typically  lead  to  a  sub-optimal 
solution  since  both  parties  jointly  favor 
increased  performance,  but  little  pressure 
initially  exists  to  prevent  performance  from 
dominating  the  resulting  solution.  The 
adverse  cost  and  schedule  of  a  performance 
dominated  solution  are  often  not  recognized 
until  relatively  late  in  development  when 
programmatic  or  technical  constraints 
become  evident. 

Knowledge  of  the  technical  possibility 
surface,  or  even  two-dimensional  slices  of 
the  surface,  may  be  unknown  or  highly 
uncertain  for  key  subsystems.  An  uncertain 
technical  possibility  surface  can  lead  to 
severe  program  risk  when  performance 
levels  approach  or  exceed  the  highly 
uncertain  state  of  the  art,  particularly  when 
program  cost  and  schedule  characteristics 
are  not  considered  in  an  unbiased  fashion. 
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When  ambitious  performance  requirements 
are  set  and  the  design  process  is  dominated 
by  performance,  pressure  exists  on  the 
government  and  contractor  alike  to  meet 
these  requirements,  even  if  program  cost  and 
schedule  are  adversely  affected  as  a  result. 

Government  and  contractor  motivations  to 
start  and  continue  programs  initially  bias 
downward  both  parties'  estimates  of  cost  and 
schedule  for  a  given  level  of  performance, 
leading— later  in  the  program-to  increased 
cost  or  schedule  in  order  to  meet 
performance  requirements.  The  issue  faced 
by  government  and  industry  is  how  to  adjust 
their  C,P,S  goals  to  be  consistent  with  the 
technological  possibilities  that  are  being 
revealed  to  them. 

By  the  time  the  problem  usually  is 
recognized  a  considerable  investment  has 
already  been  made  in  the  system  design. 
Hence,  by  this  time  the  flexibility  to  alter  the 
design  by  this  time  may  be  limited,  leading 
to  even  greater  cost  or  schedule  growth  than 
if  the  true  situation  had  been  recognized  at 
the  beginning  of  the  program  when 
performance  requirements  were  specified 
and  the  C,P,S  starting  point  was  set. 

Since  both  government  and  contractor  utility 
decreases  with  lower  performance,  both 
parties  may  only  reluctantly  decrease 
performance  requirements  during 
development  even  when  the  originally 
chosen  solution  is  infeasible.  Hence,  an 
unrealistic  C,P,S  starting  point  results  in  a 
C,P,S  solution  point  that  is  not  an  unbiased 
or  random  choice  between  cost, 
performance,  and  schedule,  leading  to 
increased  cost  and  schedule,  but  little 
decrease  in  performance. 

If  the  C,P,S  solution  point  lies  in  the 
infeasible  region,  such  as  point  A  in  Figure 
1,  resolution  of  this  unsustainable  situation 


requires  movement  of  the  solution  point  to 
the  north  with  generally  increasing  cost  or 
schedule  (point  B,  for  example),  to  the  west 
with  generally  decreasing  performance 
(point  C)  or  a  combination  of  the  two 
(between  points  B  and  C).  Since  the  utility 
of  both  parties  generally  decreases  with 
movement  towards  point  C  (less 
performance),  this  will  not  likely  occur 
unless  it  is  externally  mandated.  Hence, 
program  change  will  tend  to  be  towards  the 
north  rather  than  towards  the  west,  bounded 
by  the  technical  possibility  curve:  and  the 
resulting  solution  point  will  also  tend  to  be 
inefficient  (e.g.,  lying  above  this  curve,  as 
point  D). 

Other  movements  from  the  infeasible  to  the 
feasible  portions  of  the  technical  possibility 
surface  are  possible  depending  upon  the 
position  of  the  original  solution  point 
relative  to  the  technical  possibility  surface. 
The  key  here  is  that  the  range  of  movement 
from  an  infeasible  to  feasible  solution  point 
will  be  limited  by  the  relationship  of 
"desirability  regions”  to  the  technical 
possibility  surface. 

The  performance-driven  choice  process 
becomes  all  the  more  troublesome  when  the 
starting  point  is  relatively  high  up  on  the 
technical  possibility  surface.  Such  a  choice 
may  cause  large  cost  or  schedule  growth  if 
the  C,P,S  starting  point  is  unrealistic, 
particularly  if  impractical  performance 
requirements  are  not  relaxed.  The  resulting 
cost  and  schedule  growth  will  generally  lead 
to  increased  contractor  profit.  Since 
contractor  utility  increases  with  increasing 
profit,  they  have  little  reason  to  decrease 
cost  and  schedule,  particularly  when  in  a 
sole  source  role. 
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IS  THE  EVIDENCE  CONSISTENT 
WITH  THE  THEORY? 

Five  hypotheses  resulting  from  the 
microeconomic  framework  were  tested  and 
confirmed  by  a  statistical  analysis  performed 
on  a  large  sample  of  historic  DoD  programs. 
This  statistical  data  analysis  is  the  most 
extensive  analysis  ever  performed  on  DoD 
C,P,S  data.  It  also  provides  insights  into  the 
relationship  between  C,P,S  variable  trades 
for  military  program  acquisition,  and 
includes  results  using  corrected  Perry  et.  al. 
(1971)  and  Dews  et.  al.  data  sets. 

Statistical  analyses  of  C,P,S  change  data 
derived  from  DoD  major  weapon  systems 
can  help  to  determine  the  degree  to  which 
historical  data  are  consistent  with  the 
microeconomic  framework  of  military 
systems  acquisition. 

Three  different  data  sets  were  obtained  for 
the  statistical  analysis.  The  Perry  data  set 
was  originally  derived  from  surveys  and 
follow-up  visits  to  various  DoD  programs: 
it  represents  programs  with  Milestone  II  (or 
equivalent)  dates  in  the  1950s  (2  programs) 
and  1960s  (18  programs)  [3].  The  Dews 
data  set  was  derived  from  Selected 
Acquisition  Reports  (SARs),  and  represents 
programs  with  Milestone  II  (or  equivalent) 
dates  in  the  1970s  (18  programs)  [4].  The 
Conrow  data  set  was  also  derived  from 
SARs,  and  represents  programs  with 
Milestone  II  (or  equivalent)  dates  in  the 
1960s  (7  programs),  1970s  (17  programs), 
and  1980s  (6  programs).  The  overall 
combined  data  set  (1950s  through  1980s) 
included  58  programs  with  48,  52,  and  51 
programs  reporting  cost  change, 
performance  change,  and  schedule  change 
data,  respectively. 

Summary  descriptive  statistics  for  the 
overall  combined  data  set  are  given  in  Table 


1.  Sample  statistics  for  all  three  data  sets 
were  quite  similar,  but  since  the  overall 
combined  data  set  covers  a  longer  time 
period  (1958-1986)  and  includes  more 
systems,  Table  1  only  includes  results  from 
this  data  set.  The  mean,  median,  and 
standard  deviation  for  the  cost  change  and 
schedule  change  variables  are  similar  to  each 
other,  yet  noticeably  larger  than  the 
corresponding  values  for  the  performance 
change  variable. 

C,P,S  predictions,  drawn  from  the 
microeconomic  framework  discussed  above, 
were  tested  against  the  statistical  results. 

First  Hypothesis:  Because  of  government 
and  contractor  utility  preferences,  relatively 
few  systems  will  have  significant  overall 
development  phase  performance 
degradations.  In  addition,  since  many  U.  S. 
military  systems  have  performance 
requirements  set  at  or  beyond  the  technical 
feasibility  level,  relatively  few  programs 
will  have  large  gains  in  performance  from 
the  initial  estimated  level. 

The  hypothesis  cannot  be  rejected  given  the 
mean  and  median  values  for  the  performance 
change  distribution,  as  well  as  the  dispersal 
of  performance  change  values  around  the  no 
change  level  (1.00).  Both  the  mean  and 
median  values  for  the  performance  change 
distribution  were  1.00,  and  only  10  of  52 
programs  had  a  slip  in  performance  >  10 
percent  while  only  9  of  52  programs  had  a 
gain  in  performance  >  10  percent. 

In  terms  of  more  extreme  values,  only  two 
of  52  programs  had  a  slip  in  performance  of 
25  percent  or  more,  and  only  one  of  52 
programs  had  a  gain  in  performance  of  25 
percent  or  more. 
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OVERALL  COMBINED  DATA  SET  SUMMARY  STATISTICS  (*) 

TABLE  1 


Variable 

COST 

PERFORMANCE 

SCHEDULE 

Sample  size 

48 

52 

51 

Average 

1.26 

1.00 

1.24 

Median 

1.16 

1.00 

1.13 

Standard  deviation 

0.28 

0.13 

0.30 

Minimum 

0.86 

0.65 

0.75 

Maximum 

2.07 

1.42 

2.25 

Lower  quartile 

1.06 

0.92 

1.03 

Upper  quartile 

1.41 

1.06 

1.44 

Skewness 

1.24 

0.38 

1.24 

*:  Cost,  performance,  and  schedule  correspond  to  cost  change,  performance  change,  and 
schedule  change  associated  with  the  current  program  estimate  divided  by  the  initial 
Milestone  II  (or  equivalent)  estimate 


Second  Hypothesis:  Due  to  a  combination 
of  utility  preferences  and  technical 
feasibility,  the  variation  in  performance 
change  for  U.  S.  military  systems  is  likely  to 
be  smaller  than  corresponding  variations  in 
cost  and  schedule. 

From  Table  1,  the  standard  deviations  of 
changes  in  cost,  performance,  and  schedule 
are  0.28,  0.13,  and  0.30,  respectively. 
Consequently,  the  hypothesis  cannot  be 
rejected  given  that  the  standard  deviation  of 
performance  change  is  less  than  half  that  of 
program  cost  change  and  schedule  change. 

Third  Hypothesis:  Due  to  a  combination  of 
utility  preferences  and  technical  feasibility, 
U.  S.  military  systems  will  typically  exhibit 
an  increase  in  cost  and/or  schedule  during 
development. 

The  hypothesis  cannot  be  rejected  given  that 
43  of  48  programs  (90  percent)  exhibited 
cost  growth  and  that  40  of  51  programs  (78 
percent)  exhibited  schedule  slippage.  In 


addition,  a  number  of  programs  had  large 
cost  growth  or  schedule  slippage.  For 
example,  30  of  48  programs  had  cost  growth 
>  25  percent  and  20  of  51  programs  had 
schedule  slippage  >  25  percent. 

Fourth  Hypothesis:  The  shape  of  the 
development  performance  distribution  will 
likely  be  near-symmetrical,  whereas  the 
shapes  of  the  cost  change  and  schedule 
change  distributions  will  likely  be  right- 
hand  skewed.  This  is  because  the 
government  and  contractor  strive  to  meet 
performance  requirements,  while  typically 
adjusting  program  cost  and/or  schedule  to 
achieve  the  desired  levels  of  performance. 
Final  performance  achievements  can  be 
viewed  as  random  variations  around  the 
target  value. 

The  hypothesis  cannot  be  rejected  given  the 
skewness,  mean,  and  median  values  for 
these  change  distributions  in  Table  1.  A 
skewness  of  0  indicates  that  the  data  are 
symmetrically  distributed  while  a  positive 
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value  indicates  that  the  distribution  has  a 
right-hand  skew.  From  Table  1,  the 
skewness  coefficient  for  the  cost  change  and 
schedule  change  distributions  is  positive  and 
roughly  3.3  times  greater  than  that  for  the 
performance  change  distribution.  In 
addition,  the  skewness  coefficient  for  the 
performance  change  distribution  is  near  zero 
(0.38). 

The  difference  in  the  mean  and  median  for 
the  performance  change  distribution  is 
virtually  zero,  while  that  for  the  cost  change 
and  schedule  change  distributions  are  0.10 
and  0.11,  respectively.  These  results,  as 
with  the  skewness  results  mentioned  above, 
indicate  that  the  performance  change 
distribution  is  nearly  symmetrical,  while  the 
cost  change  and  schedule  change 
distributions  have  a  right-hand  skew. 

The  initially  infeasible  C,P,S  solution  point 
that  exists  for  many  systems  results  from  a 
mis-specification  of  the  technical  possibility 
surface  due  in  part  to  an  underestimation 
bias  associated  with  the  level  of 
performance  that  can  be  achieved  for  a  given 
level  of  cost  and/or  schedule.  This  together 
with  the  primary  government  and  contractor 
desire  to  meet  performance  requirements, 
while  allowing  cost  and/or  schedule  to 
increase  during  the  course  of  the  program  to 
achieve  a  feasible  C,P,S  solution  point, 
causes  the  cost  change  and  schedule  change 
distributions  to  have  means  greater  than  one, 
as  well  as  a  right-hand  skew. 

Fifth  Hypothesis:  Given  the  complex 
nature  of  the  C,P,S  trades  that  occur  during  a 
military  program,  no  simple  relationship  will 
likely  exist  between  the  Milestone  II  date 
and  C,P,S  change. 

There  was  negligible  correlation  between 
cost  change,  performance  change,  and 
schedule  change  versus  the  Milestone  II 


date.  Consequently,  the  hypothesis  cannot 
be  rejected  given  the  limited  correlation 
between  C,P,S  change  and  the  Milestone  II 
date. 

DISCUSSION  AND  CONCLUSIONS 

The  above  results  support  the  contention  that 
various  military  systems  acquisition  policy 
changes  instituted  by  the  government  have 
had  little  success  in  dealing  with  the 
underlying  issues  discussed  above  that 
contribute  to  program  cost  and  schedule 
growth. 

Although  the  average  cost  growth  for  the 
overall  combined  data  set  is  26  percent,  the 
dollar  magnitude  of  military  program  cost 
growth  is  also  important. 

It  was  only  possible  to  confidently  identify 
the  dollar  magnitude  of  cost  growth  for  the 
Conrow  30  program  data  set.  Initial  EMD 
phase  cost  estimates  for  the  Conrow  data  set 
were  compared  to  actual  values  recorded 
following  the  completion  of  this  program 
phase. 

The  resulting  total  cost  growth  of  this 
sample  was  $10.9  billion  (FY94).  When 
projected  to  all  military  development 
programs  over  the  past  35  years,  including 
Concept  Exploration  and  Definition  and 
Demonstration  and  Validation  phases  in 
addition  to  the  EMD  phase,  the  resulting 
total  development  cost  growth  is  likely 
between  $40  billion  and  $80  billion  (FY94)! 
(In  addition,  substantial  production  phase 
cost  growth  has  also  occurred  during  this 
time.  Production  phase  cost  growth  is  often, 
at  least  partially,  driven  by  the 
microeconomic  inefficiency  of  the 
development  phase  C,P,S  solution  point.) 

While  it  is  easy  to  identify  the  cause  of 
performance-related  problems  in  the  DoD 
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systems  acquisition  process,  it  will  be  much 
more  difficult  to  create  a  paradigm  shift  to 
deal  with  it  due  to  the  underlying  incentives 
that  are  inherent  in  the  institutional 
structure.  Having  a  set  of  C,P,S  analytic 
trade  tools  early  in  the  program  design 
process  may  aid  decision  makers  in  making 
rational  decisions.  However,  simply 
possessing  viable  trade  tools  will  not  be 
sufficient  to  eliminate  the  problems 
discussed  above  for  two  reasons. 

First,  all  military  programs  have  C,P,S  risk 
due  to  an  incorrectly  specified  technical 
possibility  surface,  which  results  from 
uncertainty  in  specifying  the  surface  coupled 
with  institutional  biases  and  preferences. 
Having  C,P,S  trade  tools  or  risk  assessment 
tools  does  not  guarantee  that  they  will  be 
applied  in  an  unbiased  fashion,  particularly 
since  both  the  government  and  contractors 
routinely  underestimate  the  risk  present  in 
military  programs  [10]  [11]. 

In  the  worst  case  the  estimated  program  risk 
is  intentionally  biased  downward  in  an 
attempt  to  portray  the  program  in  a  more 
favorable  light  with  higher  level  government 
personnel.  In  effect,  there  is  typically  little 
or  no  reward  in  most  military  programs  for 
identifying  relatively  high  risk  items  in  the 
early  development  phase  and  furnishing  the 
information  to  higher  level  government 
personnel,  since  it  increases  the  odds  of 
adverse  impacts  on  personnel,  funding 
cutbacks,  or  program  cancellation. 

Second,  the  government  and  contractor 
decision  makers  are  typically  "graded"  much 
more  strictly  on  whether  or  not  program 
performance  objectives  are  met  versus 
meeting  cost  and  schedule  objectives. 

Identifying  and  managing  prospective  cost 
drivers  and  trading-off  schedule  in  an 
attempt  to  manage  risk  is  currently  required 


by  DoDD  5000.1,  but  it  is  insufficient  given 
the  distorted  nature  of  military  program  cost 
and  schedule  versus  performance 
preferences  that  exist  [12]. 

The  wide  spread  cost  and  schedule  problems 
caused  by  the  development  of  performance- 
driven  military  systems  will  not  be 
eliminated  by  new  acquisition  trends,  such 
as  concurrent  engineering  and  total  quality 
management,  unless  they  are  specifically  re¬ 
oriented  to  address  the  changes  needed  to 
correct  underlying  government  and 

contractor  institutional  biases,  preferences, 
and  interaction  dynamics,  coupled  with  an 
uncertain  technical  possibility  surface. 

In  summary,  placing  equal  or  near  equal 
emphasis  on  meeting  program  cost  and 
schedule,  as  well  as  performance  objectives, 
will  be  necessary  in  order  to  eliminate  the 
strongly  ingrained  government  and 

contractor  program  management  bias 

favoring  performance  that  has  existed  for  at 
least  the  last  35  years  in  U.  S.  military 
systems.  This  is  particularly  important 
given  the  substantial  dollar  magnitude 
associated  with  cost  growth  during  this  time. 
No  appreciable  change  in  program  outcomes 
is  likely  to  occur  without  a  re-oriented  C,P,S 
emphasis. 

The  economic  theory  and  statistical  results 
in  this  paper  directly  address  government 
and  contractor  preferences,  interactions,  and 
outcomes  for  the  United  States  DoD  systems 
acquisition  process.  The  same  preferences, 
interactions  and  outcomes  also  exist  for 
highly  classified  ("Black")  programs,  as  well 
as  NASA  space  programs,  given  the 
structure  of  their  acquisition  process.  For 
example,  the  GAO  found  "no  major 
difference  between  the  cost,  schedule,  and 
performance  results  of  the  special  access 
acquisition  programs  it  sampled  and  those  of 
non-special  access  DoD  programs"  [13]. 
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The  GAO  also  found  considerable  cost  and 
schedule  growth  for  mature  NASA  space 
vehicle  programs  [14].  Similar  preferences, 
interactions,  and  outcomes  also  likely  exist 
for  performance-driven  large-scale  civilian 
programs  (e.g.,  megaprojects)  [15]  [16]. 

The  economic  theory  presented  in  this  paper 
can  be  adapted  to  represent  the  military  or 
commercial  systems  acquisition  process  of 
other  countries.  For  example,  the 
microeconomic  framework  has  been  applied 
to  the  former  Soviet  Union  military  systems 
acquisition  process.  Extension  of  the 
microeconomic  theory  to  predict  likely 
systems  acquisition  outcomes  for  other 
countries,  such  as  Japan  or  Sweden,  can  be 
readily  accomplished  given  a  modest 
amount  of  information. 
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ABSTRACT 


The  F-15  program  was  one  of  several 
programs  selected  to  prototype  the 
Integrated  Weapon  Systems 
Management  (IWSM)  concepts.  This 
paper  will  discuss  how  the  F-15  lead  the 
initiative  for  integrating  the  Air  Force 
Systems  Command  (AFSC)  and  the  Air 


Force  Logistics  Command  (AFLC) 
organizations  into  IWSM  under  the  new 
Air  Force  Materiel  Command  (AFMC). 
Since  the  inception  of  IWSM,  the  F-15 
has  served  as  the  benchmark  for  other 
IWSM  organizations  to  emulate  as  a 
result  of  outstanding  success. 
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F-1 5  Integrated  Weapon  System  Management 
“IWSM” 


The  Inception 

In  1991  the  F-1 5  took-on  the  role  of  lead  platform  for 
integrating  Air  Force  Systems  Command  (AFSC)  and  Air 
Force  Logistics  Command  (AFLC)  organizations  into 
Integrated  Weapon  System  Management  (IWSM)  under 
the  new  Air  Force  Materiel  Command  (AFMC).  The  two 
commands  were  previously  responsible  for  research, 
development  and  acquisition  by  AFSC  and  logistics 
support  and  sustainment  for  operations  by  AFLC.  The 
concept  of  operations  employed  for  a  successful  merger  is 
known  as  Integrated  Weapon  System  Management 
(IWSM).  Under  the  IWSM,  AFMC  presents  a  single  face 
to  the  warfighters/user  and  would  be  responsible  for 
cradle-to-grave  management  of  weapons.  The  F-1 5 
program  was  one  of  several  programs  selected  to 
prototype  the  IWSM  concepts;  the  other  programs 
included  AGM-65  Maverick,  Navstar  Global  Positioning 
System,  Joint  Surveillance  and  Target  Attack  Radar 
System  (JSTARS),  B-1B,  and  Life  Support  systems. 


The  F-1 5  senior  leadership's  IWSM  strategy  had  five 
overarching  tenets: 


a)  Bring  together  the  talents  and  capabilities  of 
both  acquisition  and  sustainment  communities  of  the 
System  Program  Office  (SPO); 

b)  Improve  business  practices  by  integrating 
management  processes; 

c)  Establish  clear  lines  of  accountability  and 
responsibility; 

d)  Institutionalize  a  cradle-to-grave  management 
concept  across  the  SPO; 

e)  Provide  a  single  face  to  our  warfighters  . 

The  senior  leadership  felt  the  key  to  satisfy  these  tenants 
was  to  organize  into  product  and  process-focused  teams, 
and  to  develop  and  execute  an  F-1 5  Master  Plan.  In 


March  1993,  the  SPO  officially  declared  IWSM  full 
operational  capability.  During  the  initial  phase,  several 
automated  tools  were  developed  and  deployed;  the  tools 
tie  the  acquisition  and  modification  programs  to  the 
Programmed  Depot  Maintenance  (PDM)/  retrofit 
programs,  and  provide  options  to  make  F-1 5  post 
production  support  strategies/decisions 

Since  the  inception  of  IWSM,  the  F-1 5  has  served  as  the 
benchmark  for  other  IWSM  organizations  to  emulate  as  a 
result  of  our  outstanding  success.  Today,  we  continue  to 
be  structured  as  one  organization,  operating  out  of  two 
geographically  separated  locations,  led  by  the  single 
manager-  the  F-1 5  System  Program  Director  (SPD).  As 
the  single  manager,  the  SPD  is  responsible  for  all  facets 
of  acquisition  and  sustainment  (cradle-to-grave 
management)  of  the  total  F-1 5  weapon  system  which 
includes  three  major  product  groups,  the  F-15A-D  (Air  to 
Air),  the  F-15E  (Air  to  Ground),  and  Foreign  Military 
Sales  (FMS).  Everything  from  nuts  and  bolts,  to  avionics, 
to  engines,  to  landing  gears,  to  support  equipment  and 
trainers-all  fall  under  the  SPDs  Area  of  Responsibility 
(AOR).  Many  of  these  items  or  subsystems  are  managed 
by  other  Product  Group  Managers  (PGMs)  or  Material 
Group  Managers  (MGMs)  located  at  other  AFMC  bases 
or  Defense  Logistics  Agencies.  For  instance,  engines  are 
managed  at  San  Antonio  ALC,  while  landing  gears  are 
managed  at  Hill  AFB.  The  bottom  line  is:  If  they  are 
attached  to  the  F-1 5,  then  the  SPD  has  oversight. 
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The  Team  Eagle 

The  SPD  is  the  single  face  to  the  customer  which  in  the  F- 
15’s  case  is  the  Combat  Air  Forces  (CAF)  which  consists 
of  Air  Combat  Command  (ACC),  USAF  in  Europe 
(USAFE)  and  Pacific  Air  Forces  (PACAF),  Air  Education 
and  Training  Command  (AETC)  and  the  Air  National 
Guard.  The  F-15  is  currently  flown  by  21  USAF  and 
National  Guard  Bureau  (NGB)  units  and  three  foreign 
countries--Saudi  Arabia,  Israel,  and  Japan.  The  United 
Arab  Emirates  (UAE)  is  also  considering  entering  the 
realm  of  the  F-15.  One  of  our  foreign  customers  (Japan) 
has  the  ability  to  produce  their  own  F-15s.  The  SPO  is 
very  sensitive  to  the  customers’  needs  and  desires  and 
therefore  maintains  very  close  communication  with  them 
on  all  aspects  of  F-15  acquisition  and  sustainment  issues. 


Prior  to  IWSM,  the  F-15  program  had  an  AFSC  office  at 
Wright-Patterson  Air  Force  Base,  which  managed  the 
acquisition  of  the  F-15E  and  new  capabilities;  while  an 
AFLC  office  at  Robins  AFB  managed  the  day-to-day 
support  and  modifications  to  the  F-15  A-D  fleet  and  two 
foreign  military  sales  offices  at  both  locations.  The 
success  of  F-15  IWSM  has  been  a  direct  result  of  our 
Integrated  Product  Team  (IPT)  and  Process  Management 
Team  (PMT)  focus. 


Even  though  SPO  North  and  South  are  currently 
organized  differently  and  in  different  geographic 
locations,  they  have  a  common  bond  which  is  the  IPT  and 
PMT  process.  These  IPTs  and  PMTs  are  made  up  of 
functional  experts  (program  management,  engineering 
hardware  and  software,  financial  management, 
manufacturing,  configuration  control,  contracting, 
production,  etc)  from  each  location.  Leadership  of  the 
IPT  or  PMT  is  dependent  upon  the  major  focus  for  that 
effort .  If  it  is  a  new  acquisition,  chances  are  SPO  North 
will  retain  the  leadership  of  the  team.  If  sustainment  is 
the  driver,  SPO  South  is  more  likely  to  be  designated  the 
lead.  Regardless  of  whether  North  or  South  leads  the 
team,  there  will  be  a  co-lead  at  the  opposite  location.  This 
ensures  that  ownership  of  the  project  is  shared  by  both 
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locations  and  all  possible  functional  specialties  have  an 
opportunity  to  provide  value  added  inputs. 


Very  important  to  Team  Eagle’s  success  is  the  team  of 
contractors,  vendors  and  suppliers  that  have  designed, 
supported,  and  redesigned  the  F-15  throughout  its 
lifetime.  The  prime  contractor  for  the  F-15  is  McDonnell 
Douglas  Aerospace  East  (MDA-E),  located  in  St  Louis 
Missouri.  In  concert  with  MDA-E  are  numerous 
significant  others  to  include:  Northrop/Grumman  with 
electronic  warning  systems  and  test  equipment;  Pratt  & 
Whitney  with  engines;  Loral,  with  avionics  subsystems; 
Hughes  for  the  radar,  and  Honeywell  and  Litton  with 
navigation  systems.  The  SPO  believes  very  strongly  that 
in  order  to  survive  the  severe  cuts  in  funding  and 
manpower  going  on  everywhere,  the  entire  F-15  Team 
(Government,  the  Warfighter,  Industry,  Educational 
Institutions)  must  work  closely  together  to  keep  the  jet 
flying.  As  a  result,  industry  has  become  an  active  and 
recognized  member  on  our  IPT  teams.  This  membership 
is  key  to  recognizing  and  acting  on,  to-date,  untapped 
resources  and  opportunities  which  must  be  exploited  to 
keep  the  Eagle  viable  in  today’s  environment. 

Continued  nurturing  of  the  SPO/Contractor  relationship 
will  hopefully  develop  improved  means  and  processes  to 
procuring  required  services  and  equipment.  Certainly,  the 
existing  system  is  much  too  oversight  intensive,  resulting 
in  high  costs  to  manage  and  field  a  system  both  in  respect 
to  time  and  funding.  Improved  relationships  with  our 


contractors  will  only  serve  to  strengthen  our  already 
outstanding  ability  to  keep  the  jet  flying. 


As  mentioned  earlier,  the  F-15  is  made  up  of  numerous 
subsystems  many  of  which  have  common  applications 
with  other  weapons  platforms.  Often  these  common 
items  or  materials  are  managed  by  a  centralized  product 
office  as  opposed  to  each  weapons  platform  managing 
their  own  requirements  and  therefore  duplicating  efforts 
and  missing  out  on  economies  of  scale.  These  product- 
oriented  organizations  are  referred  to  as  Materiel  Group 
Managers  (MGMs)  and  Product  Group  Managers 
(PGMs)-examples  of  which  are  engines,  landing  gear, 
tires,  and  fuels  just  to  name  a  few.  Because  of  the  wide 
dispersion  of  workload  throughout  AFMC,  virtually  every 
air  logistics  center  and  product  center  plays  a  major  role 
in  the  modernization,  supply  and  support  of  the  F-15. 

The  MGMs  and  PGMs  are  obviously  key  members  to 
Team  Eagle.  This  is  understandable  as  we're  sure  you 
have  already  figured  out-F-15s  don't  fly  nearly  as  long  or 
land  as  well  if  they  don’t  have  engines,  fuel  or  tires. 

As  single  manager,  the  F-15  SPD  must  stay  in  touch  with 
all  the  systems  and  products  associated  with  continued 
support  of  the  eagle.  The  SPD's  main  forum  for 
discussion  of  these  issues  is  through  semi-annual  System 
Supportability  Reviews  (SSRs).  In  this  forum,  PGMs  and 
MGMs  are  invited  to  discuss  any  problems  experienced 
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on  their  programs  where  SPD  involvement  or  knowledge 
is  required.  In  many  cases,  the  SPD  can  communicate 
these  issues  to  higher  headquarters  more  effectively  than 
the  MGM  or  PGM.  Therefore,  it  is  extremely  important 
that  both  the  MGM/PGM  and  SPD  are  telling  the  same 
story  as  to  the  health  of  a  particular  system  or  project. 
Forums  with  high  levels  of  attention  to  which  the  SPD 
has  access  include  the  Weapon  System  Performance 
Assessment  Reviews  (WSPAR)  and  the  Combat 
Operational  Readiness  Reviews  (CORR),  each  given 
yearly. 

Modernization  efforts  are  located  at  other  product  centers 
but  provide  new  capabilities  to  the  F-15.  For  example, 
Electronics  System  Center  (ESC)  provides  the  latest 
mission  planning  capability  and  electronic  warfare  items; 
while  the  Space  Systems  Center  (SSC)  provides  Global 
Positioning  System;  Human  Systems  Center  (HSC) 
provides  upgraded  aircrew  equipment  capabilities. 

In  short,  MGMs  and  PGMs  play  a  very  important  role  in 
keeping  the  Eagle  flying  everyday.  Our  continued  team 
interaction  and  communication  is  fundamental  in  keeping 
this  capability  going  strong.  The  F-15  SPO  is 
aggressively  pursuing  enhanced  partnerships  and 
improved  communications  as  integrated  product 
development  (IPD)  continues  to  grow. 


supportability.  These  organizations  work  on  the  leading 
edge  of  technology  that  is  so  critical  to  the  success  of  an 
electric  jet  such  as  the  F-15.  As  older  technologies 
become  obsolete,  newer  technology  applications  become 
critical  in  order  to  ensure  continued  operational  capability 
of  the  jet.  One  method  to  stay  abreast  of  the  emerging 
technologies  is  through  the  technology  planning  IPT,  or 
TPIPT.  The  TPIPT  briefs  program  offices  on  a  regular 
basis  to  either  gain  support  for  a  research  effort  or  gain 
insight  from  the  SPO  about  their  new  problems.  This 
process  is  healthy  for  all  involved;  it  now  includes  the 
warfighter-who  supports  those  technologies  that  bridge 
gaps  in  the  users  mission  area  plans.  Other  virtual  team 
members  include  Air  Force  Flight  Test  Center  (AFFTC), 
AEDC,  and  the  Air  Force  Development  Test  Center 
(AFDTC).  That’s  TEAM  EAGLE! 


CONCEPT  OF  OPERATIONS 


m  Integrated  product  teams  (66)  and  process  management 
teams  (5) 

-  Across  all  disci  plines/a  11  locations 

-  Based  upon  PMD  direction 

-  All  IPT  projects  baselined 

-  A11  PM  Is  with  plans 
■  Master  Plan 

-  Rolls  up  IPT  baselines 

-  Includes  major  process  plans/master  schedules 

-  Basis  for  SPO  operations/reportiug/metrics 


Single  Manager  Concept  of  Operations 
(CONOPS) 

Remember,  the  F-15  SPD  is  the  single  point  of  contact  for 
the  customer  and  is  the  single  voice  for  the  F-15  SPO,  but 
can  delegate  within  the  SPO  community  to  interface  with 
the  various  Centers/Commands  to  support  the  F-15 
mission. 


When  the  team  was  formed,  business  practices  and 
processes  needed  to  reflect  the  single  management 
concept.  Process  Management  Teams  (PMTs)  were 
formed  to  define,  review,  document  and  continually  fine 
tuned  the  critical  processes.  There  were  currently  five 
PMTs  for  Programmed  Depot,  Post-Production  Support, 
Product  Improvement, Commodity  Class  Consignment,  & 
Hazardous  Materials. 


One  of  our  quiet  and  little  known  team  members  are  the 
many  private  and  Government  Laboratories  and 
Universities  which  play  an  important  role  in  F-15 
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To  date,  15  key  Single  Manager  processes  have  been 
mapped.  The  processes  are  provided  to  the  workforce 
through  monthly  orientation  classes  taught  by  process 
owners  and  experts. 


SINGLE  MANAGER  KEY 
PROCESSES 


COST  ESTIMATION 
^  OBLIGATION  FORECAST 
BUDGET 

~  COST  MANAGEMENT 
^  SCHEDULING 
~  ECP/CCP 

^  HUMAN  RESOURCES  MGT 
SYSTEMS  ENGINEERING 
^  SYSTEM  FIELDING 
BASELINES 

/ 

z 


The  F-15  SPD  recognized  that  doing  business  in  the  new 
IWSM  environment  required  changes  at  all  levels  of  Air 
Force.  One  of  the  first  opportunities  to  make  change  was 
to  replace  the  old  Program  Management  Directive  (PMD) 
with  one  that  reflected  an  integrated  weapon  system 
management  (IWSM)  approach  for  the  development, 
support  and  continued  enhancement  of  F-15A-E  combat 
capability,  readiness,  and  sustainability.  The  PMD 
outlines  current  and  planned  products  and  projects. 
Specific  products/projects  contained  in  the  F-15  PMD  are 
used  to  build  the  F-15  APB.  Therefore,  the  next 
challenge  was  the  Integrated  Acquisition  Program 
Baseline  (APB).  The  APB  contains  key  performance, 
schedule,  and  cost  parameters  which  are  the  basis  for 
satisfying  an  identified  mission  need.  It  is  designed  to 
enhance  program  stability  and  control  cost  growth.  The 
APB  is  a  summary  and  does  not  provide  detailed  program 
requirements  or  content. 
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However,  Product/Project  Baselines  do  contain  detailed 
information  about  the  project,  such  as  cost,  development 
and  installation/retrofit  schedules,  technical  performance 
measures  that  address  the  various  “cradle-to-grave 
activities”;  other  affected  subsystems,  contrators,  and  IPT 
members.  All  IPTs  are  responsible  for  baselining  their 
efforts.  The  baseline  is  the  management  tool  to  define  the 
project  and  to  provide  traceability  to  the  F-15  APB,  and  is 
reviewed  and  scrubbed  by  the  configuration  control  board 
before  being  approved  by  the  SPD.  The  baselines  are  a 
contract  between  the  SPD  and  the  IPT.  With  an  approved 
baseline,  an  IPT  is  empowered  to  manage  and  execute  it. 
This  project  ownership  produces  the  synergy  that  creates 
a  seamless,  single  management,  IWSM  organization. 


Teams  make  it  happen. 

The  IPTs  will  maintain  close  control  of  the  most 
important  and  time  sensitive  elements  within  the  program. 
As  stated  earlier,  IPTs  are  the  foundation  of  F-15  single 
management  concept.  IPTs  bring  together  the  required 
functional  development  and  support  expertise  by  working 
among  and  across  organizational  lines  to  execute  early 
development,  modification,  or  support  programs.  IPTs 
follow  the  principles  of  integrated  product  development. 
Their  focus  is  on  a  specific  development,  integration  and 
support  product  which  demands  the  expertise  of  many,  if 
not  all,  functional  areas  at  both  the  product  and  logistics 
centers  as  well  as  appropriate  membership  for 
organizations  external  to  the  SPD. 

Recently,  a  new  IPT  was  created  to  instill  robust,  clear 
communications  (across  all  product  and  process  teams) 
from  cradle  to  grave  to  ensure  compatibility  of  all 
physical,  functional,  and  program  interfaces  in  a  manner 
that  optimizes  the  total  weapon  system  performance.  The 
Systems  Integration  Product  Team  (SIPT)  will  also 
promote  systems  integration  advancement  through 
continued  process  definition  and  improvement.  The  SIPT 
owns  the  single  manager’s  key  processes  and  serves  as 
the  “real  estate  agent”  and  “traffic  cop”  for  the  SPD. 

They  will  transform  operational  needs  into  a  description 
of  system  requirements  and  allocate  those  requirements  to 
optimize  the  overall  system  effectiveness.  The  SIPT’s 
principle  customers  are  the  product-focused  IPTs  who 
have  ultimate  responsibility  to  execute  their  baselines. 
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Another  type  team  was  formed  at  the  depot  operations  at 
Robins  Air  Force  Base.  They  are  known  as  work  center 
quality  teams  (WCQTs)  as  well  as  a  “natural  team” 
because  they  are  made  up  of  a  supervisor  (Team  Leader) 
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and  all  employees  (Team  Members)  that  are  assigned  to 
that  supervisor.  A  WCQT  works  on  problems  or 
improvement  opportunities  of  its  choice  within  its  process 
(work  environment)  by  identifying  constraints  and 
working  to  improve  quality,  increase  productivity, 
enlarge  capacity,  decrease  scrap  and  rework,  and/or 
reduce  cost.  A  WCQT  will  always  be  active.  Once  the 
team  has  achieved  an  objective  and/or  submitted 
recommendations,  it  will  initiate  another  objective 
(continuous  improvement  within  the  process). 
Membership  is  automatic  and  results  from  being  assigned 
to  applicable  work  center. 

An  excellent  example  of  Virtual  SPO  Teaming  was  the 
Desert  Eagle  Team  that  was  formed  during  Desert  Shield/ 
Storm  to  solve  severe  F-15E  supportability  problems. 

This  team  was  made  up  of  members  from  SPO  (N&S), 
ACC,  LG  staff  and  the  Senior  Leadership  of  out  prime 
contractors  (MDA,  Hughes,  ETC.)  and  chaired  by  the 
SPD.  Under  the  umbrella  of  this  team  were  sub-teams 
working  issues  like  radar  supportability,  targeting  pod 
shortages,  weapons  release  problems,  windshield  erosion, 
and  electonic  countermeasure  software.  The  pay  off  was 
record  high  aircraft  availability,  mission  capability  and 
combat  effectiveness. 

Tools 

The  set  of  automated  tools  to  aid  in  the  “cradle-to-grave” 
management  of  the  F-15  is  known  as  the  EAGLE 
Toolbox.  The  Toolbox  contains  several  automated  tools 
that  enhance  our  ability  to  manage  our  depot  retrofit 
program,  to  make  informed  post  production  support 
(PPS)  decisions  and  etc.  Depot  Retrofit  Event  and 
Modification  System  (DREAMS)  provides  on-line 
visibility  on  each  F-15  aircraft's  PDM  workload, 
modifications,  and  repair  and  configuration  status. 

Our  PPS  tools  are  the  greatest  thing  since  “crushed  ice”. 
One  cool  tool  is  Tooling  Disposition  Decision  System 
(TDDS)  uses  prime  manufacturers  data  to  match  each 
part  with  the  production  tooling,  then  provides  a  post 
production  support  recommendation  on  the 
retention/disposition  on  the  tooling.  The  Brother  tool  to 
TDDS  is  TIMS,  or  Tooling  Inventory  Management 
System,  which  ensures  that  all  production  tooling  has  an 
account  that  addresses  the  tooling  function,  usage  rates 
for  spares,  location,  storage  requirements. 

The  cradle  tools  in  EAGLE  Toolbox  support 
requirements  identification,  program  and  item 
management  and  program  execution.  The  initial 
requirements  identification  process  is  supported  by  the 
Modification  Initiatives  Management  System  (MIMS) 


which  documents,  displays,  and  supports  the  Quality 
Function  Deployment  (QFD)  process  that  establishes 
priorities  for  product  improvement  initiatives.  Automated 
Program  Management  System  (APMS)  allows  project 
managers  to  define  and  report  on  their  critical  program 
milestones,  key  parameters,  and  funding  requirements  and 
disconnects.  Candidate  Parts  List  (CPL)  is  the  repository 
and  database  for  all  F-15  parts  data,  to  include  National 
stock  Number,  Part  Number  and  Cage  code  for  current 
and  future  support  capability. 

EAGLE  Toolbox  has  received  a  lot  of  notoriety  of  late. 
Boxscore  winner  at  WR-ALC  in  Nov  94,  one  of  General 
Yates’,  Commander  of  AFMC,  “Great  Ideas”  for  1994 
and  a  selected  topic  at  the  Fall  Single  Managers 
Conference.  The  Navy  also  thought  it  was  a  great  idea 
and  are  in  the  process  of  importing  several  of  the  tools  to 
their  PPS  plans  for  the  EA-6B  and  F-14  weapon  systems. 

DREAMS,  MIMS  and  APMS  are  invaluable  when  we 
prepare  to  get  buy-in  from  the  warfighters  on  our 
programmed  depot  maintenance  (PDM)  schedules.  There 
are  two  significant  mandatory  forums  where  we  present 
our  schedules  and  work  packages  to  our  customers  and 
HQ  AFMC  representatives— the  Maintenance 
Requirements  Review  Board  (MRRB)  and  Logistics 
Support  Review  (LSR). 

At  the  MRRB,  we  present  the  organic  and  contractor 
workload  that  we  envision  for  the  next  two  years.  These 
work  requirements  define  the  tasks  that  are  to  be 
accomplished  during  PDM,  Analytical  Condition 
Inspections,  speedlines  or  by  contractor  field  teams.  The 
process  is  a  continuos  two-year  projection  of  workload 
that  is  refined  annually  by  the  board;  the  packages  are 
reviewed  on  a  quarterly  basis  The  MRRB  is  chaired  by 
our  customer  with  HQ  AFMC  serving  as  an  advisor.  The 
result  is  an  approved  workload  package  for  the  two 
subsequent  years. 
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G079  FLOW  PROCESS  CHART 


STAFF  LFC 


The  LSR  is  conducted  yearly  and  is  led  by  our  customers 
with  Air  Staff  and  HQ  AFMC  overseeing  the  process. 

Our  major  command  customers  now  have  budgeting 
responsibility  for  our  Depot  Purchased  Equipment 
Maintenance  (DPEM)  requirements.  At  the  review, 
DPEM  requirements  from  the  G079-- System  & 
Equipment  Modification/Maintenance 
Program(SEMMP)~system  are  validated.  The  workload 
requirements  are  based  on  the  DREAMS  (schedule), 
MRRB  hours,  and  approved  hourly  rates.  The  LSR  looks 
at  a  three  year  window  of  requirements  and  allows  the 
customers  to  submit  their  requirements  into  the  three  year 
Financial  Plan  for  the  Budgeting  Estimate  Submission  to 
Congress.  This  process  provides  the  ALC  the  dollars 
necessary  to  perform  the  workload,  whether  it  be 
contract/organic.  Depot  Field  Team  (DFT)/Contract  Field 
Team  (CFT)  workload. 

One  year  ago,  the  F-15  SPO  initiated  a  forum  known  as 
the  Aircraft  Configuration  Management  Review 
(ACMR).  The  ACMR  is  the  result  of  a  fighter  wing’s 
realization  that  the  downtime  associated  with 
accomplishing  Time  Compliance  Technical  Orders 
(TCTOs)  adversely  impacted  their  flying  program 
objectives.  Team  Eagle  analyzed  the  circumstances  and 
reengineered  their  TCTO  process-instituted  the  ACMR 
process.  The  ACMR  takes  a  18-month  forward  look  at 
the  TCTO  efforts,  to  include  validation  and  verification 
milestones  for  those  programs  in  the  queue. 
Superintendents  from  each  fighter  wing  and  HQ  ACC 
logisticians,  along  with  our  production  planners, 
logistics/program  managers  review  and  synergize  TCTO 
work  packages.  Even  though  the  process  is  young,  it  has 
already  reaped  dividends  through  improved 
communications  with  the  warfighters  about  aircraft 
downtime. 


In  the  forementioned  cases,  the  warfighters  get  the 
opportunity  to  review  our  work  packages,  rates  and 
schedules  before  hand,  work  the  hard  issues  at  the  grass 
roots  level  and  we  obtain  relatively  easy  buy-in  by  senior 
leadership. 


All  good  teams  have  a  vision,  the  team  developed  the 
following: 


F-15  TEAM  EAGLE  VISION 


•  The  21st  Century  F-15  Eagle  will  be  operationally  effective, 
affordable,  and  sustainable  in  support  of  the  Air  Force  Vision 
and  U.S.  National  Security  objectives. 

•  The  F-15  will  always  be  a  fighter  that  has  the  total 
confidence  of  the  warfighter  allowing  the  Eagle  to  fly,  to  fight, 
to  win,  and  to  return  home— every  time. 

•  The  people  of  Team  Eagle  care  about  each  other.  We 
maintain  the  highest  standards  of  safety  and  quality  from  start 
to  finish— because  what  we  do  is  important.  In  spirit  we  ail  fly 

mlk  the  Ewte  gff  imsirn 


E1S  EAGLE  D4ILY  GOAL 


Team  Eagle  Strives  each  day  to  develop,  s  ustain,  maintain,  and  fly 
the  highest  quality  fighter  at  the  lowest  possible  cost 


F-15TEAM  EAGLE  MOTTol 


“The  BesLEighteL- ..anytime,  anywhere!” 


From  top  to  bottom,  PMD  to  APB  to  Baselines  to  IPTs  to 
vision, goal  and  motto,  the  F-15  program  had  made  the 
cultural  transition  and  others  looked  on  to  follow  their 
lead.  Teams-more  so  PEOPLE  are  the  principal  tool  the 
SPD  has  for  ensuring  that  the  F-15  weapon  system 
satisfies  all  user  requirements.  The  bottomline  is  that  the 
foundation  for  effective  continuous  improvement  and 
reengineering  is  a  real  and  demonstrated  love  and  respect 
for  people. 
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At  a  recent  strategic  offsite,  Team  Eagle  decided  to 
construct  their  future.  They  envisioned  the  jet  being  in 
the  inventory  until  2020.  They  also  recognized  that  the 
Team-SPO,  Contractors,  MGMs/PGMs  and  etc.,  would 
have  to  do  more  with  less  resources.  And  that  the  people 
that  would  be  flying,  maintaining,  developing  and 
supporting  the  weapon  system  would  definitely  change,  - 
many  of  whom  are  not  yet  bom! 

EAGLE  FUTURE 


EAGLE  GENERATION 


EAGLE  EAGLE 

ENTERPRISE  UNIVERSITY 


The  brainchild  of  this  thinking  became  Eagle  Future. 
Eagle  Future  is  comprised  of  three  distinct,  but  mutually 
inclusive  elements:  Eagle  Generation,  Eagle  Enterprise, 
and  Eagle  University. 

Eagle  Generation  is  the  roadmap  we  are  following  to 
ensure  the  F-15  will  remain  an  effective  fighter  well  into 
the  21st  century.  We  are  working  hard,  everyday,  toward 


this  end  and  our  challenge  is  to  institutionalize  the  process 
so  that  it  becomes  our  everyday  philosophy.  Eagle 
Generation  was  a  sucess  story  for  the  F-15  program -the 
process  we  employed  set  the  stage  for  HQ  Air  Combat 
Command  to  develop  the  Fighter  Configuration  Plan 
(FICOP).  The  FICOP  was  used  to  prioritize  the  funding 
for  modernization  of  the  fighter  force  structure.  Once 
again,  the  F-15  model  of  doing  business-strategic 
planning— gets  a  kudo.  The  Eagle  Generation  process  is 
being  institutionalized  and  will  continue  to  be  used  to 
construct  the  sustainment  roadmap  for  the  jet. 

Eagle  Enterprise  is  our  plan  to  integrate  and  meld  all  F- 
15  people,  from  the  users  to  the  product  centers,  the 
ALCs,  contractors,  and  research  universities  into  one 
cohesive  team.  The  result  is  “Team  Eagle”—  a  VIRTUAL 
ORGANIZATION  whose  common  purpose  is  to  make 
the  F-15  the  best  fighter.  To  date,  we  have  removed 
geography  and  time  from  the  equation  through  our 
information  super  highway  between  our  two  SPO 
locations,  PGM/MGMs  and  several  key  contractors.  We 
can  actually  access  data  located  at  a  remote  location,  say 
El  Segundo  CA,  review  and  comment  on  bales  of  data 
right  at  our  desk-saves  shipping  and  travel  dollars.  We 
also  keep  our  warfighters  informed  about  supportability 
issues  via  our  Eagle  Bulletin  Board  or  e-mail. 

Eagle  University  (EU)  is  the  recognition  that  we  need  a 
“learning  organization”  to  keep  a  “core  competency” 
within  Team  Eagle.  EU  provides  Covey  leadership 
training,  Theory  of  Constraints,  and  process  training.  EU 
is  also  the  “Keeper  of  the  Flame”-the  story,  the  legacy. 
The  F-15  has  quite  a  legacy— in  the  past,  you  would  have 
heard  aircrews  say  proudly  “The  F-15-we  go  where  others 
have  never  gone  before!” 


As  you  can  see,  IWSM  starts  and  ends  with  the  warfighter 
focus-cradle-to-grave  management  by  a  single 
management  team,  TEAM  EAGLE,  that  is  accountable 
for  all  aspects  of  the  F-15  weapon  system. 
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GOVERNMENT  PROPERTY  - 

A  CRITICAL  RESOURCE  IN  THE  ACQUISITION  PROCESS 

Douglas  N.  Goetz,  Ph.D. 

The  Air  Force  Institute  of  Technology 


ABSTRACT 

The  field  of  Government  property  has  made 
front  page  headlines  numerous  times  and  has 
been  the  subject  of  numerous  General 
Accounting  Office  (GAO)  reports  to 
Congress  as  well  as  the  subject  of  untold 
Inspector  General  (IG)  reports  from  all 
agencies,  civilian  and  military.  The  field  of 
Government  property  and  its  use  in  the 
acquisition  process  must  be  addressed  for 
improvements  to  be  reaped.  This  paper  will 
analyze  the  current  policy,  application  in  the 
acquisition  process,  and  changes  on  the 
horizon  with  the  rewrite  of  the  Government 
property  part  of  the  Federal  Acquisition 
Regulation. 

INTRODUCTION 

"While  data  we  obtained  showed  that  as  of 
September  30,  1984,  contractors  had  over 
$8.4  billion  of  GFE  [Government  furnished 
equipment]  in  their  possession,  there  are 
known  problems  concerning  the  accuracy  of 
this  data. "  1 

"DOD  furnishes  billions  of  dollars  of 
property  to  defense  contractors  each  year. 
However,  the  exact  amount  of  property 
currently  in  the  hands  of  contractors  is 
unknown."  2 

As  of  September  1986  there  was  over  $45 
billion  of  Government  property  reported  in 
the  possession  of  defense  contractors.  3 


As  of  September  1993  there  was  over  $90 
billion  of  Government  property  reported  in 
the  possession  of  defense  contractors.  4 

Government  property  has  been  the  brunt  of 
many  jokes  in  the  acquisition  community. 
"Do  you  know  what  the  two  types  of 
Government  property  are?  Late  and 
defective! "  Yet,  if  one  were  to  more  closely 
analyze  the  impact  of  Government  property  in 
the  acquisition  process  one  would  realize  that 
its  role  is  critical  and,  at  this  point  in  our 
procurement  history,  indispensable.  In  point 
of  fact,  the  irrefutable  claim  could  be  made 
that  if  all  Government  property  were 
immediately  withdrawn  from  availability  this 
action  would  cause  the  shutdown  of  every 
major  weapon  system  acquisition.  Think 
about  it.  Do  any  of  the  weapon  systems 
you're  involved  with  use  Government 
property?  Let's  look  at  some  of  the  most 
frequent  applications. 

For  Aircraft  -  Engines  as  Government 
furnished  property, 

For  Naval  Ships  -  Radar,  sonar  systems, 
missile  guidance  systems  as  Government 
furnished  property, 

For  Army  Tanks  -  Treads,  fire  control 
systems  as  Government  furnished  property, 
Special  Tooling  -  to  fabricate  all  of  the  above, 
as  Government  furnished  property, 

Special  test  equipment  -  to  test  all  of  the 
above,  as  Government  furnished  property, 
Facilities,  i.e.,  general  puipose  plant 
equipment  and  real  property  (Don't  believe 
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me?  Check  out  our  GOCOs.)  to  produce  the 
actual  deliverable. 

These  are  just  a  few  examples  of  the  use  of 
Government  property  in  the  acquisition  of 
DOD  weapon  systems.  But  they  do  serve  to 
illustrate  the  critical  nature  of  the  use  of 
Government  property  in  defense  procurement 
and,  to  some  extent  the  overuse  and  over 
dependance  upon  the  same.  To  analyze  the 
use  of  Government  property  one  must  first 
look  at  the  Government's  policy  on  providing 
Government  property  to  defense  contractors 
or,  for  that  matter,  any  contractor  doing  work 
for  the  federal  government. 

POLICY  ON  PROVIDING 

GOVERNMENT  PROPERTY 

The  Federal  Acquisition  Regulation  (FAR)  is 
quite  specific  in  its  opening  reference  of,  Part 
45,  the  Government  Property  part.  It  states 
"Contractors  are  ordinarily  required  to  furnish 
ALL  (emphasis  added)  property  necessary  to 
perform  government  contracts. "  Yet,  in  spite 
of  the  policy  we  have  seen  the  amount  of 
Government  property  in  the  possession  of 
DOD  contractors  burgeon  to  over  $90 
billion.5  _  Without  a  doubt,  there  has  been  a 
dramatic  increase  in  the  amount  of 
Government  property.  There  may  be  many 
reasons  why  this  has  happened,  increased 
defense  budgets  over  the  decade  preceding 
this  last  report,  cost  avoidance  through  the 
furnishing  of  Government  property,  lack  of 
knowledge  as  to  the  application  of  policy 
exceptions  when  the  Government  may 
provide  Government  property;  the  list  is 
endless. 

POLICY  EXCEPTIONS 

Rather  than  rationalize  it  would  be  better  to 
review  first  the  policy  exceptions  when 
Government  property  may  be  furnished  to  a 


contractor.  These  exceptions  are  not  as 
succinctly  stated  or  logically  organized  as  the 
original  policy  statement.  One  must  search 
and  cull  them  out  from  the  FAR,  Department 
of  Defense  Federal  Acquisition  Regulation 
Supplement  (DFARS),  numerous  Directives, 
Instructions,  and  Manuals,  as  well  as  civilian 
agency  regulations.  These  policy  exceptions, 
for  providing  Government  property  to 

contractors,  consist  of  the  following: 

1.  To  achieve  significant  economy, 

e.g.,  Precious  metals  recovered  by  the 

Government  or  items  from  a  breakout 
program,  a  jet  engine. 

2.  Standardization,  e.g.,  Special 
tooling  for  First  Article  acceptance  and 
inspection  testing  or  for  production  use. 

3.  To  expedite  production,  e.g., 

Long  lead  items  such  as  forgings  or  machine 
tools. 

4.  To  increase  competition,  e.g., 

Special  Test  equipment  to  allow  the  small 
business  to  compete. 

5.  To  support  Small  Business,  e.g., 
Special  tooling  for  the  production  of  spares. 

6.  To  maintain  the  industrial  base, 
e.g.,  Industrial  plant  equipment  to  meet  surge 
capacities  at  an  ammunition  plant. 

7.  Due  to  the  scarcity  of  an  asset, 
e.g.,  Strategic  materials. 

8.  Lastly,  the  type  of  contract 
awarded,  Fixed  Price  versus  Cost 
reimbursement  type  contracts,  as  all  property 
acquired  by  the  contractor  under  a  Cost 
reimbursement  contract  title  vests  in  the 
Government. 


COSTS  AND  RISKS  ASSOCIATED  WITH 
PROVIDING  GOVERNMENT  PROPERTY 

Those  these  previously  mentioned  items  may 
seem  easy  to  understand;  they  are,  in  fact, 
deceptively  complex.  Yes,  these  are  the 
policy  exceptions  for  providing  Government 
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property.  But,  those  involved  with  the 
acquisition  process  must  be  aware  that  there 
are  costs  and  risks  associated  with  the  action 
of  providing  Government  property  that  are 
not  elaborated  upon  in  the  policy  exceptions. 

Let's  look  at  some  of  the  costs  and  risks 
associated  with  providing  Government 
property. 

Timely  delivery  and  suitable  for  use 

One  of  the  most  critical  problems  with 
providing  Government  property  hinges  upon 
the  "guarantees"  the  Government  makes 
regarding  its  suitability  for  use  and  timely 
delivery.  If  we  allowed  the  contractor  to 
acquire/fabricate  all  property  necessary  to 
produce  the  deliverable  end  item  the 
contractor  bears  the  risk  of  ensuring  that  the 
item  does,  in  fact,  work.  When  we,  the 
Government,  furnish  any  of  the  components 
to  be  incorporated  into  a  deliverable  end  item 
the  Government  property  Clauses  stipulate 
that  the  GOVERNMENT  is  responsible  if  the 
item  of  Government  furnished  property 
(GFP)  is  not  suitable  for  its  intended  use  and 
the  Government  is  also  liable  if  the  item  is 
not  delivered  on  a  timely  basis.6  In 
commercial  contracting  you  will  not  find  an 
equivalent  process  of  "providing"  property  to 
a  contractor.  Yet,  we  in  the  Government 
furnish  property  on  a  daily  basis.  Remember 
the  over  $90  billion  out  there?  If  that 
property  is  furnished  in  a  condition  not 
suitable  for  its  intended  use  or  in  a  timely 
fashion  the  Government  is  required  to  provide 
an  equitable  adjustment  to  the  contractor.  So 
in  essence  the  Government  has  paid  for  the 
Government  property  originally,  furnished  it 
to  a  contractor,  and  is  now  paying  again 
because  it  didn't  work  right  or  was  delivered 
late.  Surely  there  must  be  a  better  way. 


Loss.  Damage  and  Destruction 

Assuming  that  the  property  got  there  on  time 
and  in  a  condition  that  was  suitable  for  its 
intended  use  what  could  happen  next?  Well, 
if  everything  goes  okay  the  end  item  is 
delivered.  Sure,  how  often  does  everything 
go  okay?  Well,  suppose  for  a  moment  that 
the  contractor  loses  that  properly,  or  it  is 
damaged  or  destroyed?  Well,  most 
Government  property  clauses  provide  for  the 
contractor  to  be  granted  relief  of 
responsibility  if  the  contractor  has  an 
approved  property  control  system.7  If  such 
is  the  case,  where  the  contractor  does  not  bear 
the  risk  of  loss  for  that  Government  property, 
it  is  rather  simple  to  see  who  is  going  to  have 
to  pay  for  the  replacement  of  that 
Government  property.  As  much  as  it  may 
anger  us,  the  Government  will. 

So  let's  see.  The  Government  furnishes  the 
property  -  its  late  or  unsuitable  for  use,  the 
Government  pays  the  contractor  to  make  it 
suitable  or  for  any  delay  in  production.  The 
property  breaks,  or  is  lost,  or  is  damaged, 
and  the  Government  pays  for  its  repair.  Not 
a  bad  deal  if  your  the  contractor.  A  real 
bummer  if  you're  the  Government.  I  realize 
that  I  am  engaging  in  hyperbole  and  speaking 
in  the  pejorative  but,  do  you  really  want  to 
furnish  that  property  to  the  contractor? 

Ahhhh,  but  there's  more.  Assume  that 
everything  goes  okay.  Property  is  delivered 
on  time,  its  suitable  for  use,  and  there  is  no 
loss  damage  or  destruction.  That's  the  end  of 
the  Government's  costs  for  this  property 
right?  Wrong.  Because  the  contractor  must 
also  establish  records,  care  for  and  maintain 
that  property,  physically  inventory  it  and 
report  those  results  to  the  Government.  All 
of  these  actions  add  additional  costs  to  the 
contractor's  overhead,  thereby  increasing 
your  procurement  costs. 
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These  are  just  a  few  example  of  the  increased 
costs  and  risks  the  Government  bears  when  it 
assumes  the  responsibility  of  providing 
Government  property 

CONCERNS 

In  1986  former  Undersecretary  of  Defense 
Richard  Godwin  issued  a  memo  to  all 
secretaries  of  military  departments  and 
directors  of  defense  agencies  to  reduce  the 
amount  of  Government  property  provided  to 

g 

defense  contractors. 

This  memo  contained  direction  in  nine  areas 
consisting  of: 

Industrial  facilities, 

Incentives  for  contractor  investment 
Special  tooling/Special  Test  equipment 
Material 

Property  rollovers/transfers 

Property  to  service  contractors 

Storage  of  Government  property 

Disposal/retention  of  Government  property 

and 

General  Management  areas 

I  would  like  to  take  a  moment  and  discuss 
some  of  the  areas  found  in  this  still  applicable 
policy  letter. 

Industrial  facilities 

Put  quite  bluntly,  the  Government  should  not 
be  furnishing  nor  should  it  be  authorizing 
contractors  to  acquire  for  the  Government 
general  purpose  machinery  or  equipment. 
Contractors  are  expected  to  have  their  own 
general  purpose  equipment  for  performance 
of  the  contract.  Even  more  importantly,  we 
expect  contractors  to  invest  their  own  funds  in 
the  purchase  of  general  purpose  equipment. 
An  analogy,  you  contract  to  have  your  two 
story  house  painted,  the  painter  arrives  ready 


to  go  and  asks,  "O.k.,  where  are  the 
ladders?"  An  assumption  is  made  that  the 
painter  has  all  of  the  necessary  tools  and 
equipment  to  do  the  job.  Yes,  the 

Government  will  pay  for  the  material  used  in 
performing  the  contract,  either  through 
progress  payments  (financing)  on  fixed  price 
contracts,  or  through  reimbursement  on  cost 
reimbursement  type  contracts.  The 
Government  will  pay  for  the  "special"  stuff, 
i.e.,  special  tooling  and  special  test 
equipment.9  But  it  expects  the  responsive 
contractor  to  have  all  of  the  general  puipose 
"stuff'  necessary  to  perform  the  contract. 
This  action,  providing  general  purpose 
equipment  to  the  contractor,  does  nothing  but 
take  acquisition  dollars  that  could  be  spent  for 
research,  development  or  production  of  an 
end  item  and  diverts  them  to  facilitizing  the 
contractor. 

Material 

There  has  been  a  long  standing  belief  that 
material  available  through  the  Government 
supply  system  and  Government  inventories  is 
stuff  that's  free.  The  Godwin  memo 
emphasized  that  the  Government  should  not 
be  furnishing  commercial,  off-the-shelf 
material  as  Government  furnished  property. 
Why?  Because  one  must  again  analyze  all  of 
the  record  keeping  and  usage  requirements 
and  restrictions  imposed  upon  the  contractor 
for  the  control  of  Government  property. 
Contractor  should  be  acquiring  commercial, 
off-the-shelf  material  through  their  own 
purchasing  system.  I  can  hear  some  of  you 
now.  "But  the  Material  Management 
Accounting  System  (MMAS  allows  the 
contractor  to  commingle  and  use  material 
from  cost  reimbursement  type  contracts, 
progress  payments  financed  material  and  even 
contractor  owned  material  -  so  what's  the 
problem?"10  You  are  absolutely  correct. 
The  MMAS  was  developed  to  allow  the 
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contractor  to  make  maximum  utilization  of 
the  inventory  available  regardless  of 
ownership.  With  one  exception  -  remember 
that  commercial,  off-the-shelf  material  you 
FURNISHED  the  contractor?  Well,  it  is 
considered  Government-furnished  property 
(material)  and  CANNOT  be  included  in  the 
MMAS. 

Disposal  or  retention  of  Government 
property 

Let  me  take  you  one  other  area  in  the 
acquisition  process  -  final  disposal  or 
retention  of  Government  property  at  contract 
completion  or  phase  out  of  a  weapons  system. 

Remember  all  that  "stuff”  you  furnished  the 
contractor?  Well,  the  contract  is  over,  the 
contractor  doesn't  want  it  in  his/her  plant 
anymore  because  some  other  competitor  was 
awarded  the  follow-on  contract.  And  oh,  by 
the  way,  the  follow-on  contract  awardee 
doesn't  want  the  Government  property 
because  his  equipment  uses  different 
processes.  The  Government  is  now 
responsible  for  disposing  of  those  assets. 
Disposition  can  be  a  rather  lengthy,  expensive 
process  and  the  monies  necessary  will  be 
coming  from  procurement  funds  better  spent 
on  the  product.  [Note:  This  is  even  more 
critical  if  the  contractor  uses  any  type  of 
material  which  may  become  a  hazardous 
waste.]  Lastly,  in  this  area  is  the  issue  of 
retention  of  Government  property,  most 
notably,  special  tooling.  For  years 
procurement  folks  used  the  dreaded  NO- 
COST  STORAGE  AGREEMENT.  This 
item  is  in  essence  an  oxymoron.  There  are 
always  costs  associated  with  storage  of  any 
type  of  property.  In  this  instance  there  were 
no  direct  costs.  Rather,  the  costs  were  loaded 
into  overhead,  thereby  increasing  the 
contractors  costs  across  ALL  contracts, 
Government  and  commercial.  Yes,  the 
logisticians  are  going  to  want  to  keep  every 


item  of  special  tooling  ever  created  for  the 
weapons  system.  There  exists  the  pack  rat 
mentality  that  we  will  need  it  one  day.  How 
many  of  you  are  collecting  those  National 
Geographies  and  will  read  them  all,  one  day? 
A  tough  call  must  be  made  as  to  the  probable 
future  need  of  the  special  tooling!  This  call 
must  take  into  consideration  the  cost  of 
storing  the  items  for  X  number  of  years,  the 
administrative  costs  associated  with 
controlling  those  items,  the  costs  of  shipping 
to  another  location  (multiple  times),  the 
technical  obsolescence  of  the  tooling,  i.e., 
does  the  equipment  on  which  the  tooling  was 
used  still  exist?  The  real  question  -  "Is  it 
cheaper  to  store  or  cheaper  to  buy  new  ten 
years  down  the  road?"  That  is  a  very  tough 
question  to  answer. 

These  are  just  a  few  areas  where  those 
involved  in  the  acquisition  of  weapons 
systems  or  of  any  item  for  the  Government 
need  to  be  aware  of  the  impact  and  costs 
associated  with  the  providing  of  Government 
property. 

ALTERNATIVES  TO  PROVIDING 
GOVERNMENT  PROPERTY 

In  spite  of  the  numerous  reports,  the  increase 
of  Government  property  already  in  the  hands 
of  contractors  there  is  some  light  at  the  end  of 
the  tunnel  -  and  acquisition/procurement 
officials  can  help  in  this  area. 

Action  items 

STOP  PROVIDING  GOVERNMENT 
PROPERTY!  Realistically,  that  won't 
happen  because  there  are  always  reasons 
when  it  will  make  clear,  good  economic  sense 
to  do  so.  What  I  ask  is  that  when  you  do 
decide  to  provide  Government  property  that 
you  ensure  that  it  is  in  compliance  with  the 
specific  policy  directives.  And,  to  reiterate: 
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•  Do  not  authorize  contractors  to 
acquire  general  purpose  equipment  (Including 
computers)  without  the  specific  approval 
requirements  set  forth  in  the  FAR. 1 1 

•  Do  not  furnish  commercial,  off- 
the-shelf  material  from  the  Government 
supply  source  without  an  economic  analysis 
of  the  cost  benefit  to  the  Government  taking 
into  consideration  the  previously  discussed 
costs. 

•  Be  prepared  to  promptly  dispose  of 
residual  assets  AND  to  pay  for  the  costs 
incurred  for  the  disposal  of  hazardous  wastes. 

•  Understand  the  guarantees  that  the 
Government  makes  regarding  Government- 
furnished  property,  i.e.,  suitable  for  intended 
use  and  timely  delivery  otherwise  contractor 
is  entitled  to  an  equitable  adjustment. 

•  Understand  the  liability  provisions 
for  Government  property  and  that  the 
Government  generally  acts  as  a  self  insurer. 

FUTURE  TRENDS 

By  the  time  you  read  this  paper,  the  rewrite 
of  the  Federal  Acquisition  Regulations  will 
have  been  underway  for  a  period  of  time. 
The  Government  property  section  was 
selected  by  Ms.  Eleanor  Spector,  Director  of 
Defense  Procurement,  as  one  of  the  first  parts 
to  be  redone.  To  rewrite  this  portion  of  the 
FAR  is  a  difficult  balancing  act.  Specifically 
it  requires  the  application  of: 

Best  commercial  practices  -  What 
does  industry  do  with  their  stuff?  How  do 
they  control  their  assets? 

Balancing  cost  efficiency,  risk  and 
responsiveness  to  the  taxpayer  and  the 
taxpayer’s  dollar. 

Compliance  with  law  and  statute 
governing  certain  Government  property 
control  situations. 


In  response  to  the  numerous  GAO  and 
DODIG  reports,  the  rewrite  committee  must 
take  action  to  prevent  the  proliferation  of 
Government  property.  The  simplest  way  to 
accomplish  this  is  to  tighten  up  the  policy  on 
providing  Government  property.  This  course 
of  action  is  already  in  the  works.  The  policy 
on  providing  Government  property  will 
become  more  restrictive.  The  committee  also 
acknowledges  that  it  would  be  imprudent  to 
close  and  lock  the  door  entirely.  There  are 
always  valid  reasons,  economy,  expediency, 
standardization,  etc.,  that  call  for  the 
providing  of  Government  property.  The 
issue  is  acquisition  personnel  MUST  be  aware 
of  that  policy  and  make  economically 
defendable  decisions  in  compliance  with  that 
policy. 

ENDNOTES 

1  GAO/NSIAD-86-109  Government 
Equipment.  Defense  should  further  reduce 
the  amount  it  furnishes  to  contractors. 

2  GAO/NSIAD-88-151  Government 
Property.  DOD’s  management  of  the 
property  it  furnishes  to  contractors. 

Source:  Contract  Property  Management 
System  (CPMS)/DD  Form  1662  as  of  30 
Sep  Each  year. 

4  Source:  Contract  Property  Management 
System  (CPMS)/DD  Form  1662  as  of  30 
Sep  Each  year. 

5  Source:  Contract  Property  Management 
System  (CPMS)/DD  Form  1662  as  of  30 
Sep  Each  year. 

6  FAR  52.245-2(a)  and  -5(a)  Both  stipulate 
that  the  Government  warranties  that  the 
Government-furnished  property  is  both 
suitable  for  use  and  will  be  delivered  on  a 
timely  basis. 
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7  The  Govememnt  property  clauses  found 
at  52.245-2(Alt.I),  52.245-5(g)  and 
52.245-8  all  stipulate  that  to  hold  a 
contractor  liable  the  Government  must 
prove  willful  misconduct,  lack  of  good 
faith  on  the  part  of  contractor’s  managerial 
personnel  or  have  a  disapproved  property 
control  system.  There  is  extensive  case  law 
and  writings  on  that  subject  including 

ASBCA  No.  29,831.  Dynalectron 
Corporation,  July  31,  1985. 

ASBCA  No.  214387.  Fairchild 
Hiller  Corporation,  Nov  30,  1971. 

g 

Office  of  Under  Secretary  of  Defense 
(OUSD/A&L(PS))letter  dated  25  Nov 
1986. 

9  FAR  52. 232- 16(d)  and  52.245-5(c) 

10  Department  of  Defense  Federal 
Acquisition  Regulation  Supplement 
(DFARS)  242-7004  and  252.242-7004 

11  FAR  52.244-2  and  45.302. 
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APPLYING  HARDWARE  PREFERRED  PARTS  QUALIFICATION  CONCEPTS 
TO  THE  ACQUISITION  OF  COMMERCIAL  SOFTWARE 

Theresa  Ridge  Huber,  DSD  Laboratories,  Inc. 


ABSTRACT 

The  DoD  is  changing  how  systems  are 
developed  by  moving  towards  investing  in 
dual  use  products  and  processes  and  in 
increasing  use  of  commercial  standards, 
specifications,  and  products  [1].  Utilizing 
commercial  products  to  meet  DoD 
requirements  will  help  reduce  the  acquisition 
cycle  time,  improve  quality  and  consistency 
of  delivered  products,  and  as  a  result, 
reduce  risks.  However,  to  implement  this 
new  direction  for  acquiring  commercial 
software  products,  software  acquisition 
managers  need  to  know  not  only  how  to 
obtain  the  appropriate  products,  but  also 
how  to  determine  if  and  under  what 
circumstances  the  commercial  products  meet 
the  DoD’s  requirements.  To  answer  these 
questions  for  acquisition  managers,  we  can 
start  by  examining  the  qualification  process 
for  hardware-related  parts,  which  can  easily 
be  applied  to  software  products. 

This  paper  details  how  qualification  concepts 
for  hardware  preferred  parts  can  be  applied 
to  the  acquisition  of  commercial  software 
products,  by  examining  the  concepts  and 
vision  of  establishing  a  centralized 
organization  to  acquire,  qualify  and 
distribute  commercial  software  products  for 
systems  in  similar  application  areas.  This 
vision  and  associated  concepts  described  in 
this  paper  were  developed  jointly  by  the  Air 
Force’s  Comprehensive  Approach  to 
Reusable  Defense  Software  (CARDS) 
[Contract  #F19628-93-C-0130]  and  NASA’s 
Software  Optimization  and  Reuse 
Technology  (SORT)  -  Reuse  Adoption  and 
Techniques  Team  (SRATT)  [Contract  # 
WVHTC-F-S94-1008]  Programs,  with 


support  from  the  the  Air  Force’s  Portable, 
Reusable,  Integrated  Software  Modules 
(PRISM)  Program. 

INTRODUCTION 

There  are  various  software-related  concerns 
in  the  DoD:  software  systems  take  too  long 
to  develop  (costs  and  schedules  are 
underestimated);  costs  of  software 
development  are  increasing,  and  at  the  same 
time  the  quality  of  initially  delivered 
software  is  not  sufficiently  high  [2].  To 
address  some  of  these  concerns,  the 
Secretary  of  Defense  has  issued  a  directive 
[1]  to  move  towards  the  use  of  industry 
specifications  and  standards.  The  goal  of 
this  mandate  is  to  streamline  and  simplify 
the  way  DoD  purchases  systems  by 
removing  military-unique  specifications, 
providing  access  to  state-of-the-art 
commercial  technology,  and  developing  non- 
Govemment  standards  in  partnership  with 
industry.  This  ruling  provides  a  preference 
for  commercial  items  for  Government- 
unique  requirements  and  removes 
impediments  to  using  commercial  products. 
In  fact,  utilizing  commercial  software  in 
system  development  can  result  in  significant 
benefits.  However,  several  concerns  have 
been  raised  from  the  Program  Management 
and  acquisition  perspectives:  where  are 
components  found?;  how  are  components 
determined  suitable  for  a  particular 
application?;  and  how  does  one  address  the 
increased  schedule  and  expense  of 
conducting  preliminary  analyses? 

To  address  these  concerns,  a  central 
organization(s)  could  be  designated  to 
conduct  preliminary  analyses  to  determine 
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needs  of  similar  systems,  acquire  the  needed 
commercial  software  products,  examine  the 
products  for  suitability  for  systems  within  an 
application  area,  and  distribute  those 
products  to  Program  Offices  and  system 
developers.  One  solution  is  to  apply 
hardware  preferred  parts  concepts  and 
current  software  reuse  practices  [3]  to  the 
purchase,  qualification  and  distribution  of 
commercial  software  products. 

For  hardware  parts,  qualification  is 
performed  in  advance  of  need  and 
independently  of  any  specific  acquisition. 
The  names  of  successful  products, 
manufacturers  or  potential  offerors  are 
included  on  lists  showing  their  status.  There 
can  be  several  sets  of  criteria  for  which  the 
part  is  evaluated  against.  For  example,  the 
evaluations  can  be  based  on  a  Program 
Office’s  criteria  for  a  particular  application 
or  they  can  be  based  on  industry 
standards/ specification  s . 

The  process  of  qualifying  hardware  parts 
based  on  certain  specifications  and 
standards,  and  then  using  those  pre-qualified 
parts  during  development,  can  easily  be 
adapted  to  software  [4].  The  policies  and 
procedures  in  the  Federal  Acquisition 
Regulation  (FAR)  Part  9.2,  which  govern 
the  qualification  of  hardware-related  parts, 
is  valid  and  applicable  to  such  a  concept  for 
software  components  [4]. 

The  remainder  of  this  paper  details  the 
vision  and  associated  concepts  to  establish  a 
non-profit  organization  to  centrally  acquire, 
qualify  and  distribute  commercial  software 
products  for  systems  in  a  similar  application 
area  (e.g.,  command  centers)  or  product  line 
(e.g.,  PRISM  -  a  specific  architectural 
approach  for  command  centers).  This 
approach  will  establish  a  central  source  for 
Program  Offices  and  system  developers  to 
obtain  commercial  software  products  and 


information  on  the  use  of  those  products 
within  the  particular  application  area.  A 
consistent  verifiable  approach  to  qualifying 
components  will  be  provided  to  the 
customers  of  this  centralized  facility  [2]. 

This  centralized  qualification  facility  will  be 
a  source  of  proven,  reliable  components  for 
incorporation  into  architectures  for  common 
systems  and  will  accelerate  the  development 
or  modification  of  those  systems.  This 
effort  supports  DoD  policy  changes  to  use 
commercial  products  and  components  by 
not  only  making  those  software  products 
available,  but  by  conducting  analyses  that 
are  required  prior  to  requirements  analysis. 
This  concept  helps  meet  the  DoD’s  goals  of 
increasing  use  of  commercial  products  and 
standards,  which  will  allow  for  more 
effective  and  efficient  evaluation  and 
maintenance  of  "existing"  components. 

COTS  SOFTWARE  QUALIFICATION 
CONCEPT  OF  OPERATIONS 

To  reiterate  from  above,  the  central 
organization  will  acquire  the  needed 
commercial-off-the-shelf  (COTS)  software 
products,  qualify  the  products  for  suitability 
for  common  applications,  and  distribute 
those  products  for  use  by  Program  Offices 
and  system  developers  who  develop  systems 
for  the  particular  application  area  (e.g., 
command  centers).  Figure  1,  Component 
Qualification  Process,  represents  the  overall 
process  [2],  A  commercial  product  is 
acquired  (or  submitted)  for  qualification,  the 
product  is  qualified,  placed  on  a  list,  and 
through  the  use  of  various  delivery 
mechanisms  is  made  available  to  Program 
Offices  and  system  developers.  There 
would  be  a  different  set  of  qualification 
criteria  for  each  application  area  or  product 
line. 

The  major  participants  in  the  process  are  the 
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Acquisition  Organization,  Commercial 
Software  Vendors,  and  the  COTS 
Qualification  Facility,  which  are  illustrated 
in  Figure  2. 

As  Figure  2  shows,  the  acquisition-related 
responsibilities  is  separated  into  3  groups: 
users,  Program  Offices/System  Developers, 
and  Product  Line  Organization  [3]. 

The  user’s  role  here  is  the  same  as  the 
traditional  role  of  stating  a  need  and 
generating  requirements.  The  Product  Line 
Organization  manages  the  use  and  reuse  of 
software  across  systems  within  a  particular 
application  area.  Included  in  this  role  is  the 


product  line  engineering  team  which 
conducts  the  preliminary  analyses  and 
develops  the  generic  requirements  for  the 
group  of  systems.  Based  on  the  analysis, 
the  product  line  engineering  team  develops 
and  maintains  the  software  architecture  and 
the  evaluation  criteria  that  will  be  used  by 
the  COTS  Qualification  Facility  to  evaluate 
products  for  use  by  system  developers.  The 
Program  Offices  and  System  Developers 
utilize  the  information  supplied  by  the 
Product  Line  Organization  to  assist  them 
developing  systems  based  on  common 
architectures  and  commercial  products. 

The  commercial  software  vendors  provide 
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(Non-Profit)  (Government) 


Figure  2:  Major  Roles 


their  products  to  the  facility  for 
qualification.  The  current  plan  is  to  follow 
the  hardware  model,  where  the  vendor 
would  pay  for  the  qualification,  unless  they 
are  an  8(a)  company.  The  vendors  obtain 
the  qualification  results  from  the  COTS 
Component  Qualification  Facility  to  guide 
revisions  and  updates  of  products.  These 
results  are  also  provided  to  customers  when 
distributing  the  products  through  their  own 
distribution  channels. 

For  this  concept  to  be  successful,  the  COTS 
Qualification  Facility  will  be  operated  under 
federal  sponsorship  by  an  independent,  non¬ 
profit  organization.  The  facility  must 
demonstrate  no  conflicts  of  interest.  They 
must  not  be  in  the  business  of  developing 
systems  in  the  chosen  application  areas  and 
will  be  prohibited  from  competing  for  work 
in  which  the  qualified  products  are  used. 
The  facility  will  also  be  required  to  show  a 
capability  to  safeguard  sensitive  vendor 
information. 


As  illustrated  in  Figure  3,  the  COTS 
Qualification  Facility  has  4  major  roles: 
management  of  the  process,  acquiring  the 
products,  qualifying  those  products,  then 
distributing  them  to  appropriate  users.  The 
management  role  covers  both  the  internal 
management  of  the  entire  process  and 
interactions  the  organization  has  with 
industry  as  a  whole.  The  acquisition  role 
pertains  to  the  qualification  facility’s 
interaction  with  COTS  vendors.  The 
qualification  role  pertains  to  the  qualification 
facility’s  interaction  with  the  Product  Line 
Engineering  Team.  The  distribution  role 
pertains  to  the  qualification  facility’s 
interaction  with  Program  Offices/System 
Developers.  These  roles  are  described 
below. 

COTS  QUALIFICATION  FACILITY 

To  establish  a  successful  organization  and  a 
market  for  the  qualification  and  distribution 
of  COTS  products,  the  COTS  Qualification 
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Facility  must  interact  and  collaborate  with 
all  the  major  players,  as  well  as 
Government,  industry  and  academic  interest 
groups,  to  get  feedback  and  participation. 
The  widest  possible  participation  is  needed 
up-front  in  the  development  of  the 
qualification  criteria  and  the  product 
qualification  and  distribution  processes  to 
ensure  agreement  and  buy  in  by  stakeholders 
in  the  market.  COTS  Vendors  would  be 
interested  in  reviewing  and  commenting  on 
the  submittal,  qualification  and  distribution 
processes.  They  would  primarily  be 
interested  in  safeguarding  sensitive  or 
proprietary  data.  Since  Program  Offices 
and  System  Developers  will  rely  on  the 
suitability  and  acceptability  of  the  products 
for  use,  they  would  be  interested  in 
reviewing  and  commenting  on  the 
qualification  process  and  distribution 
processes.  The  COTS  Qualification  Facility 


will  need  to  establish  relationships  with  the 
various  chosen  product  line  organizations  to 
establish  the  process  of  providing  feedback 
on  qualification  results. 

Advertisements  and  announcements  of  the 
creation  of  the  COTS  Qualification  Facility 
and  the  general  concepts  behind  it,  will  be 
disseminated  utilizing  various  marketing 
efforts:  brochures,  conference  presentations, 
internet  discovery  tools,  such  as  Mosaic 
Home  Page,  Government  electronic  bulletin 
board  systems,  and  the  Commerce  Business 
Daily  (CBD). 

Various  stakeholders  will  be  solicited  to 
review  and  provide  comments  on  the 
different  processes  mentioned  above.  First, 
announcements  will  be  made  explaining  the 
concept  of  operations  of  the  facility.  Then, 
the  background  material  will  be 
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disseminated  for  use  in  two  workshops  to  be 
held  to  draft  the  operational  processes  (one 
for  technical  and  one  for  business).  The 
workshop  results  will  be  draft  qualification 
criteria  and  business  arrangements  for  the 
facility  to  use  as  a  baseline. 

In  the  short  term,  the  Government  will 
expense  initial  capital  and  operating  costs 
through  an  agreement  which  supports  the 
establishment  and  operation  of  the  facility. 
In  the  long  term,  the  COTS  Qualification 
Facility  will  become  self-sustaining.  The 
facility  will  charge  vendors  fees  for 
qualifying  products  and  will  charge  the  users 
of  products  a  small  fee  for  distribution  of 
products  and  the  qualification  results.  The 
fees  will  cover  the  recurring  costs  of  the 
process,  but  will  be  reasonable  enough  to 
encourage  vendors  to  submit  COTS  products 
for  qualification. 

Some  type  of  policy  is  needed  so  that  system 
developers  are  either  encouraged  or  required 
to  utilize  qualified  COTS  products.  To 
ensure  a  market  for  components,  it  is 
assumed  that  the  Government  will  eliminate 
contractually  required  testing  of  that 
component.  An  alternative  is  to  require 
system  developers  to  demonstrate  why  a 
particular  product  can  not  be  used.  Any 
request  should  address  requirements,  risk, 
cost  and  schedule.  This  will  in-tum  ensure 
that  there  is  a  market  for  the  products  and 
encourage  vendors  to  submit  their  products. 
Thus,  the  Government  needs  to  provide  a 
clear  perception  to  vendors  so  that  their  ROI 
predictions  will  justify  their  participation 
and  will  increase  the  acceptance  and  use  of 
their  products  in  the  marketplace  [2]. 

In  interacting  with  COTS  vendors,  the 
COTS  Qualification  Facility  must  establish 
a  means  to  solicit  vendors  to  submit 
products  and  develop  formal  agreements  to 
describe  the  business  relationships. 


COTS  vendors  will  be  solicited  by  various 
avenues.  These  include:  soliciting  Requests 
for  Information  using  the  Commerce 
Business  Daily,  making  requests  directly  to 
vendors,  establishing  and  conducting 
working  groups  with  vendors,  and  by  using 
currently  established  Government  electronic 
bulletin  boards. 

Formal  agreements  will  need  to  be 
established  with  the  various  vendors  to 
document  the  terms  and  conditions  of  the 
business  relationships.  Agreements  will 
cover  product  submittal  process  issues,  such 
as  feedback  on  results  of  the  qualification, 
an  appeal  process,  if  the  vendor  does  not 
agree  with  the  qualification  results,  how 
product  upgrades/enhancements  will  be 
treated,  and  protection  of  proprietary  data. 
Other  terms  and  conditions  of  the 
agreements  will  cover  how  to  distribute 
products  to  end-users  and  the  qualification 
policies/procedures . 

It  is  extremely  important  to  choose  an 
application  area  that  has  a  strong  foundation 
of  commercial  products  (i.e.,  a  market 
already  exists),  so  that  it  will  be  cost 
effective  to  establish  the  infrastructure  and 
support  processes. 

The  first  application  area  will  be  command 
centers  and  the  first  product  line  will  be  the 
Air  Force’s  generic  architecture  from  the 
Portable,  Reusable,  Integrated  Software 
Modules  (PRISM)  Program.  The  command 
center  application  area  has  functional  areas 
common  to  the  Management  Information 
Systems  (MIS)  application  area  and  has  a 
large  base  of  COTS  products  (e.g. ,  Database 
Management  Systems,  Network  Managers, 
Geographic  Information  System,  and 
Message  Translator/Validator).  In  addition, 
command  centers  is  one  area  in  the  DoD 
that  are  continually  being  upgraded  to  reflect 
new  technologies. 


178 


A  set  of  qualification  criteria  will  be 
developed  for  product  lines  within  an 
application  area  (e.g.,  the  PRISM  generic 
architecture  for  command  centers).  The  set 
of  qualification  criteria  will  apply  generally 
to  the  product  and  for  each  function  within 
the  application  area  (e.g.,  geographic 
information  system,  message 
translator/validator).  If  one  product  fits  into 
two  different  application  areas,  then  it  would 
be  evaluated  twice,  using  the  two  sets  of 
qualification  criteria.  If  the  product  fits  into 
two  different  functional  areas  within  a 
product  line,  then  it  is  also  evaluated  twice, 
once  for  each  function. 

Products  will  be  qualified  against  product 
line  requirements  and  architecture 
constraints,  both  on  a  stand  alone  basis,  and 
through  integrating  the  product  into  a 
prototype. 

The  COTS  Qualification  Facility  must  work 
closely  with  the  Product  Line  Engineering 
Team  to  ensure  that  continual  feedback  is 
provided  on  the  results  of  qualifications,  so 
that  the  architecture  and  associated 
qualification  criteria  are  properly 
maintained. 

Once  the  COTS  products  are  qualified,  the 
results  will  be  disseminated  to  all  parties 
involved:  the  COTS  vendor,  the  Product 
Line  Engineering  Team,  and  the  Program 
Offices/System  Developers.  The 
information  to  be  disseminated  will  include 
a  description  of  the  product,  the  stand  alone 
qualification  results,  and  the  analytical 
results  of  how  the  product  integrates  with 
the  other  products  in  the  architecture.  The 
COTS  vendor  would  use  the  information  for 
future  enhancements  of  the  product,  as  well 
to  include  with  the  product  for  any  future 
direct  sales.  Program  Offices/System 
Developers  not  only  need  access  to 
qualification  results,  but  also  need  a 


capability  to  purchase  the  commercial 
products.  The  COTS  Qualification  facility 
will  provide  mechanisms  for  Government 
agencies  and  their  contractors  to  purchase 
these  qualified  COTS  products. 

There  are  many  possible  channels  and 
mechanisms  to  use  for  distributing  COTS 
products  and  qualification  results  that  may 
be  feasible  to  use  (See  Figure  1).  Products 
may  be  available  thorough  the  GSA,  directly 
from  COTS  vendors,  identified  and 
accessible  through  reuse  libraries  (e.g., 
Asset  Source  for  Engineering  Technology, 
(ASSET),  CARDS  Command  Center 
Library,  Defense  Software  Repository 
System  (DSRS),  and  the  Army  Reuse 
Center),  or  a  contract  that  contains  these 
components  (e.g.,  an  8(a)  or  other  suitable 
mechanism).  Products  could  even  be 
obtained  directly  from  the  qualification 
center  if  it  could  be  done  without  the 
appearance  of  conflict  of  interest  (i.e.,  the 
qualifier  is  not  compromised  by  also  being 
a  non-exclusive  source  of  qualified 
components  [2]). 

CONCLUSIONS 

This  concept  of  a  centralized  facility  to 
acquire,  qualify  and  distribute  commercial 
products  to  Program  Offices  and  system 
developers  is  valid,  needed  and  valuable  to 
meeting  the  DoD’s  directive  to  increase  the 
use  of  commercial  software.  To  summarize, 
the  facility  will  collect  results  of  preliminary 
analyses  from  participating  product  line 
engineers  to  determine  the  need  for  certain 
commercial  products.  Based  on  analyses 
performed  by  product  line  engineers,  the 
facility  will  acquire  the  needed  commercial 
software  products  and  examine  them  for 
applicability  to  systems  within  an  application 
area.  The  facility  will  also  distribute  those 
products,  and  associated  qualification 
results,  to  Program  Offices  and  system 
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developers. 

There  are  two  major  challenges  confronting 
this  effort.  The  first  challenge  is 
establishing  a  market  for  qualified 
component.  This  challenge  is  to  establish  a 
demand  by  the  customers  (acquisition 
managers  and  system  developers)  for  the 
commercial  software  products.  Having  a 
customer  base  will  provide  an  economic 
incentive  to  vendors  to  submit  their  products 
for  qualification  [2],  The  second  challenge 
is  the  capability  of  applying  commercial 
software  products  in  Government  system 
development  efforts  [5].  This  brings  up 
many  sub-issues,  such  as  how  the  software 
will  be  maintained,  determining  the 
suitability  of  COTS  software  for  use  in 
mission-critical  systems,  and  obtaining 
support  (documentation  &  maintenance)  by 
vendors  for  the  life  of  the  system  [6]. 

Given  that  these  challenges  can  be  met, 
there  are  several  benefits  to  this  strategy. 
By  utilizing  pre-qualified,  existing  products 
and  the  results  of  the  product  line  analyses, 
the  development  cycle  time  can  be  reduced 
and  the  quality  and  reliability  of  systems  can 
be  improved.  Further,  training  and 
maintenance  costs  can  be  reduced  because  of 
enhanced  familiarity  with  and  improved 
reliability  of  reused  components  [1].  This 
strategy  also  improves  quality  and 
consistency  of  delivered  products,  as  well  as 
establishing  core  competencies  around 
strategic  business  interests. 
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ACQUISITION  STREAMLINING 
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ABSTRACT 

Statistics  can  be  used  in  a  variety  of  ways. 
An  integral  element  of  Total  Quality 
Management  (TQM)  is  the  need  to  verify 
"facts  versus  opinions"  using  metrics  to 
isolate/minimize  emotionalism.  Statistics  can 
also  be  used  to  explain  what  might  be 
considered  an  "irrational"  stance  on  an  issue. 
Statistics  definitely  have  a  role  in  attempting 
to  quantify  risk. 

This  paper  develops  a  paradigm  to  analyze 
risk  using  simple  statistical  tools.  This 
paradigm  can  be  applied  to  decisions  within 
acquisition  circles  where  the  consequences  of 
making  a  mistake  may  be  relatively  small  or 
where  a  mistake  may  mean  life  or  death. 


technical  inspectors  (TIs)  seemed 
"irrational".  Right  behind  the  1st  rule  of 
"Light's  Laws  to  Aircraft  Maintenance"  (If  it 
ain't  broke,  don't  fix  it)  was  "Don't 
rationalize  with  irrational  people—TIs."  It 
seemed  their  orientation  on  decisionmaking 
was  different  for,  at  times,  it  seemed  they 
would  pull  perfectly  good  parts  out  of  a 
helicopter  and  replace  them.  I  later  was 
given  some  statistical  tools  at  school  that 
revealed  they  were  not  only  rational,  but 
quite  conscious  of  the  consequences  of  their 
decisions. 


INTRODUCTION 


Years  ago,  when  I  was  a  practicing  aircraft 
maintenance  officer,  I  tried  to  figure  out  why 
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While  attending  graduate  courses  at  the 
Naval  Postgraduate  School  in  Monterey, 
CA,  I  received  a  course  on  statistics. 
Normal  curves  (Bell  curves)  were  included 
routinely  in  analyzing  all  sorts  of  problems. 
One  would  set  up  a  hypothesis  and  then  go 
about  setting  a  "confidence  factor"  to  prove 
or  disprove  the  hypothesis.  "Alpha" 
represented  a  probability  and  is  shown  in 
Figure  1  as  a  "tail"  probability.  When 
calculating  whether  the  hypothesis  was 
rejected,  the  point  of  rejection  occurred 
when  the  calculation  placed  you  in  the  "tail" 
area  {The  probability  of  rejecting  an 
otherwise  true  statement). 

Now  back  to  TI  s.  Let's  statistically  examine 
why  he  decides  things  as  he/she  does.  Say 


he  assumes  a  part  or  work  completed  is 
"O.K."  and  in  reality,  it  is.  He's  right  and  he 
signs  off  the  fault  in  the  logbook.  Suppose 
he  assumes  the  part/work  is  "bad"  and  it  is— 
again,  he's  right  and  properly  rejects  the 
work  and  directs  that  it  be  redone. 

Suppose  now  he  assumes  a  part  or  work 
completed  is  "bad",  but  in  reality,  it  was  just 
fine.  He  has  committed  a  Type  Alpha  error 
and  rejected  the  initial  or  "null"  hypothesis. 
The  consequences  of  his  error  are  wasted 
time,  money,  and  effort.  Risk  in  this  area  is 
generally  considered  low.  But  take  a  look  at 
a  more  costly  error...  suppose  he/she  accepts 
parts  or  work  completed  as  "O.K."  that  are 
indeed  defective.  He  has  now  committed  a 
Type  Beta  error  and  erroneously  signed-off 
the  fault.  The  consequences  of  this  error, 
carried  to  the  extreme,  are  damaged 
equipment  or  loss  of  life. 
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■'Tail" 


FIGURE  1 


ASSUMPTION 

REALITY 

Part/Work  is  "O.K.” 

Part/Work  is  "Bad" 

Part/Work  is  "O.K." 

CORRECT:  "InspO.K." 

Type  Alpha  error: 

Wasted  $,  time,  effort 

(LOW  RISK} 

Part/Work  is  "Bad” 

Type  Beta  error: 

Potential  damage  to 

property  or  loss  of  life 

(HIGH  RISK} 

CORRECT:  Reject  repair 

&  direct  work  to  be  redone 

FIGURE  2 


So,  I  concluded  that  TIs  rationally  consequences  were  the  ones  to  be  avoided 

determined  that  the  error  with  the  worst  (Type  Beta  errors).  Therefore,  if  a  TI  had 
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any  doubts,  he  would  default  to  the  lower 
risk  error  of  rejecting  the  part/work 
erroneously  vs.  accepting  faulty  work  and 
perhaps  hurting  someone. 

As  Total  Quality  Management  (TQM)  made 
its  debut  in  acquisition  circles,  it  focused 
primarily  on  three  principles— Employee 
Involvement,  Continuous  Process 
Improvement  vs.  Product  Orientation,  and 
Metrics  to  separate  "opinions"  from  "facts". 
Simple  tools  revolved  around  normal  "bell" 
curves  and  reducing  process  variability 
around  a  mean.  As  TQM  evolved,  statistics 
started  being  used  in  acquisition  circles  to 
prove  or  disprove  assumptions.  Today,  we 
find  Requests  for  Proposals  (RFPs) 


statistically  analyzed  to  determine  whether  or 
not  they  adhere  to  acquisition  streamlining 
principles.  Well,  let’s  apply  the  Type 
Alpha/Beta  analysis  to  acquisition 
streamlining  for  a  moment. 

Assume  for  a  moment  that  as  a  general  rule, 
the  acquisition  community  will  now  accept 
the  general  proposition  that  contractors  are 
"Responsible  and  trustworthy."  By  virtue  of 
this  assumption,  acquisition  streamlining 
seems  to  make  a  world  of  sense.  Why 
impose  unnecessary  regulations  and  "how¬ 
tos"  on  a  contractor  who  would  have  done 
the  "right"  thing  anyway.  Figure  3  illustrates 
a  complementary  argument  to  the  one  of  the 
TI  s 


184 


ASSUMPTION 

REALITY 

Contractors  are 

Responsible  and 

Trustworthy 

Contractors  are  not 

Responsible  and 

Trustworthy 

Contractors  are 

Responsible  and 

Trustworthy 

CORRECT:  Acquisition 

streamlining  works,  saving 

$,  time,  and  effort 

Type  Alpha  error: 

Government  wastes  $  on 

unneeded  MILSPECS, 

MILSTDs  and  "  how-tos” 

in  contracts 

Contractors  are  not 

Responsible  and 

Trustworthy 

Type  Beta  error: 

Government  discovers  the 

consequences  of  this  error 

in  equipment  that  doesn't 

work  or,  at  the  extreme, 

people  who  die  in  combat 

or  training 

CORRECT:  Based  on  past 

performance.  Government 

appropriately  levies 

MILSPECS,  MILSTDs, 

and  "how-tos” 

FIGURE  3 


Again,  the  consequences  of  one's 
assumptions  are  most  damaging  with  the 
Type  Beta  error,  perhaps  explaining  why 
some  are  reluctant  to  totally  embrace  some 
aspects  of  acquisition  streamlining  as  it  is 
presented  today. 


CONCLUSIONS/SUMMARY 

The  pendulum  has  swung  over  the  years 
from  overly  restrictive  regulation  ($600  toilet 
seats)  to  minimal  regulation  based  on  a  "best 
value"  approach.  We,  as  a  community, 
should  not  have  to  wait  for  a  catastrophe  to 
occur  before  reaching  the  epoch  of  the 
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extremes  and  causing  the  pendulum  to  swing 
back  toward  the  center.  Project  and  product 
managers  (PMs)  must  also  be  allowed  to 
analyze  and  comment  dispassionately  on 
current  policy  without  being  accused  of  "old 
style"  thinking.  Granted,  one  must 
sometimes  turn  90  degrees  to  get  out  of  a 
rut,  but  if  that  rut  is  adjacent  to  a  cliff,  you 
may  want  to  insure  you  turn  on  the  correct 
side  of  that  rut. 

Acquisition  corps  officers  and  civilians  can 
make  life  and  death  decisions,  just  like 
combat  commanders.  The  difference  is  they 
are  insulated  from  the  immediate  effect  of 


their  decisions  because  some  decisions  do 
not  take  hold  until  years  later.  A  second 
difference  is  the  soldier  has  to  live  with  many 
decisions  for  the  rest  of  the  weapons' 
systems  life  cycle.  The  examples  cited  above 
are  simply  food  for  thought  in  the 
progressive  evolution  of  new  and  innovative 
acquisition  strategies  for  the  future. 

ENDNOTES 
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By 
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THE  PROBLEM 

Information  systems  technology 
development  is  increasing  at  a  faster  rate 
than  anyone  ever  could  have  anticipated. 
The  new  product  introduction  cycle  for 
personal  computers  is  less  than  a  year.  This 
is  not  to  say  that  these  products  are  unusable 
in  a  year,  but  increased  capabilities  and 
features  are  constantly  being  offered.  The 
benefits  of  these  new  products  draw 
customers  and  diminish  the  interest  in 
existing  hardware  and  software.  Hardware 
continues  to  get  smaller  with  increased 
performance,  memory,  storage,  and  speed. 
Software  applications  offer  more  features, 
ease  of  operation,  compatibility,  security, 
and  integration.  However,  newer  software 
applications  may  not  be  backward 
compatible  with  existing  systems.  Video 
teleconferencing  (VTC)  systems  are  also 
constantly  increasing  their  products  and 
features  in  a  similar  product  introduction 
cycle.  However,  the  video  conferencing 
industry  has  not  yet  established  a  universal 
standard  which  would  make  all  VTC  systems 
interoperable.  Networks  and 

communications  carriers  are  also  increasing 
performance  and  offering  new  services  and 
packages  at  a  rapid  rate.  So  many  services 
are  offered  that  it  would  be  impractical  for 
an  individual  user  to  maintain  currency  with 
the  offerings.  With  current  technology,  an 
"office"  is  no  longer  restricted  to  the 
traditional  location.  The  "office"  now 


includes  the  home,  a  hotel  room,  the  car,  the 
airport  or  anyplace  else  where  a  modem 
connection  is  available. 

Because  of  the  pace  of  technology,  the  usual 
acquisition  lead  time  for  systems  involving 
commercial  products  has  become 
unacceptable.  In  most  cases,  the  larger 
acquisitions  have  taken  up  to  two  years  to 
get  to  contract  award.  For  a  new 
requirement  involving  any  sort  of 
technology,  the  traditional  system 
development  life  cycle  would  only  be  feasible 
if  it  were  possible  to  "freeze"  requirements 
and  technology  for  about  two  years.  By  the 
time  large  computer  contracts  are  awarded, 
systems  offered  are  a  year  behind  current 
commercial  offerings.  Also,  prices  which 
may  have  seemed  attractive  at  the  start  of  the 
process  are  not  a  good  deal  shortly  after 
award.  Even  worse,  because  these  models 
may  have  been  "closed-out"  by  private 
supply  sources  the  now  obsolete  system  is 
very  difficult  for  the  customer  to  sustain. 

Organizations  have  not  been  building 
information  systems  using  a  well-planned 
top-down  design.  Acquisition  program 
offices  have  been  purchasing  "exactly  what 
the  program  office  needs."  Each  solution 
may  not  be  in  line  with  their  neighbor, 
creating  "islands"  of  automation.  Because  a 
typical  program  office  deals  only  with  a 
limited  set  of  offices  (3-4)  on  a  regular  basis, 
their  choice  of  systems  reflects  a  small  slice 
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of  the  command.  When  individual  systems 
were  put  in  place,  many  offices  had  no  idea 
what  standards  were  being  pursued,  whether 
their  efforts  duplicated  another  office’s 
efforts,  or  if  the  solution  had  already  been 
developed.  The  buying  of  systems  in 
isolation,  largely  by  individual  program 
offices  results  in  a  “bottom-up”  design  of  the 
command’s  information  system,  with  the 
interfaces  between  offices  mostly  ignored. 

To  compound  the  resulting  interoperability 
problem,  the  present  command  structure  is 
the  result  of  a  merger  of  several  separate 
commands  with  differing  information 
systems  structures  and  systems  with  only 
limited  compatibility. 

The  existing  contracting  process  itself  is  also 
a  barrier.  Large  contracts  demand  lots  of 
lead  time,  even  though  requirements  and 
technology  change.  Acquisition  panels  are 
followed  by  numerous  meetings  until  finally 
all  the  paragraphs  of  the  solicitation  are  in 
the  right  order.  The  acquisition  team  spends 
much  of  their  time  changing  the  wording  to 
reflect  the  current  reviewer's  comments. 
This  process  seems  to  have  changed  every 
time  a  new  acquisition  is  initiated.  The  result 
is  that  the  acquisition  team  gets  so  bogged 
down  with  contracting  matters,  they  lose 
sight  of  the  user’s  needs,  and  have  difficulty 
maintaining  currency  with  the  commercial 
market. 


FINDING  A  MODEL  FOR  THE 
SOLUTION 

Because  the  acquisition  system  has  become 
so  complex  and  time-consuming,  it  defies  a 
solution  without  a  radically  new  approach  to 
re-engineer  the  entire  process.  The  solution 
however  is  not  to  develop  a  more  complex 
system,  but  a  relatively  simple  system. 


A  hundred  years  ago,  the  country  store  was 
the  source  of  supply  for  many  people.  But 
the  old-time  country  store  was  more  than  a 
place  to  buy  things.  It  also  was  a  source  of 
product  information  which  many  people 
relied  upon  to  arrive  at  their  decision.  The 
country  store  also  provided  a  referral  service 
to  direct  customers  to  other  more 
appropriate  sources  of  supply.  When  a 
customer's  unique  need  could  not  be  satisfied 
any  other  way,  the  country  store  would 
obtain  the  item  for  the  customers.  A  bulletin 
board  allowed  the  customers  to  post  then- 
own  items.  The  country  store  had  constant 
feedback  from  its  customers  and  could  act 
accordingly.  The  country  store  was  not  a 
mandatory  source  of  supply,  since  it  had  to 
be  good  enough  to  bring  back  satisfied 
customers.  Being  a  central  resource,  there 
was  strength  in  numbers  and  a  vision  in 
serving  the  entire  community. 

Even  today,  the  small  New  England 
hardware  store  still  serves  a  similar  function 
in  providing  the  "value-added"  product 
information  which  one  needs  to  select  the 
right  materials  for  a  home  repair  project. 
Admittedly  the  small  hardware  store  is  a 
dying  breed  in  trying  to  compete  pricewise 
with  the  giant  "home  warehouses".  But 
although  the  "warehouse"  store  has  slightly 
lower  prices  and  more  selection,  they  may 
not  provide  product  advice.  Without 
product  information,  you  would  often  buy  a 
product  which  is  not  suitable  for  your  home 
repair  job,  and  ends  up  being  returned  to  the 
store.  My  local  New  England-style 
hardware  store  can  tell  me  exactly  what  I 
need  to  do  a  job,  and  often  how  to  do  it. 
The  storekeeper  can  even  tell  me  if  one  of 
my  neighbors  did  a  similar  project.  The  store 
gets  feedback  if  I'm  not  successful,  and 
something  has  to  be  custom  made.  Project 
success  is  less  risky  when  I  deal  with  my 
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local  hardware  store  with  their  value-added 
customer  service.  In  a  similar  way,  Air 
Force  program  managers  are  too  busy 
building  state-of-the-art  airplanes  to  acquire 
the  information  technology  product 
knowledge  necessary. 

THE  SOLUTION 

Just  as  the  old-time  country  store  served  its 
community,  the  AFMC  Country  Store  is  a 
source  of  product  information,  contract 
sources,  and  resources  to  help  end-users  to 
clarify  their  needs,  acquire  the  right 
products,  or  find  a  solution  already 
developed  and  available.  The  Country  Store 
takes  the  concept  a  step  further  in  applying 
principles  of  Total  Quality  Management, 
including  strategic  planning,  organizational 
improvement,  employee  empowerment  and 
teamwork,  primary  attention  on  customer 
needs,  and  feedback  and  measurement  of 
performance.  As  an  improvement  of  the  old- 
time  country  store,  the  AFMC  Country 
Store’s  information  is  available  on-line  and 
readily  accessible  at  any  time.  Many 
requirements  can  be  satisfied  by  locating  an 
existing  contract,  or  finding  a  software  tool 
already  developed.  The  Country  Store 
provides  listings  of  resources  to  further  help 
the  customer,  including  listings  of  expert 
help,  guidebooks,  directories  and  other 
items.  The  Country  Store  offers  guidance  on 
current  standards  and  the  state  of  technology 
for  those  products.  The  Country  Store's 
leadership  in  system  interoperability  and 
standards  can  foster  increased  connectivity 
which  will  gradually  eliminate  automation 
"islands". 

How  does  the  Country  Store  improve 
acquisition?  First  of  all,  fewer  acquisitions 
will  be  necessary  as  the  Country  Store  helps 
people  find  ready  sources  for  their 


information  technology  requirements.  This 
alone  is  an  advantage,  since  the  acquisition 
cycle  has  been  immediately  shortened.  With 
fewer  acquisitions,  resources  can  be  tasked 
to  accomplish  the  remaining  acquisitions 
with  better  focus  on  total  quality  and 
command-wide  objectives.  Second,  the 
central  listing  of  contracts  will  allow  users  to 
compare  offerings  much  easier  than 
previously  to  obtain  exactly  what  they  need 
at  the  best  price.  Through  the  Defense 
Information  Systems  Agency  (DISA), 
arrangements  are  being  made  to  be  able  to 
scan  available  contracts  across  all  services 
and  civil  agencies.  Third,  the  Country  Store 
provides  software  process  tools  approved  by 
the  process  owner  in  several  functional  areas 
of  acquisition.  Facilitating  the  use  of 
approved  tools  can  foster  the  use  approved 
best  practices.  Also,  the  Country  Store  is  set 
up  to  anticipate  requirements  through 
periodic  surveys,  and  its  ties  to  the  local 
bases.  On-line  feedback  can  be  incorporated 
into  acquisition  action  much  more  efficiently 
than  previously  unstructured  methods.  The 
Country  Store  has  packaged  complete 
solutions  to  problems  in  plain  language, 
describing  the  technology  and  where  it  can 
be  obtained.  The  Country  Store  has  offered 
such  "packaged  solutions"  on  video 
conferencing  systems,  groupware,  e-mail 
systems,  Internet  software,  and  security 
tools.  Through  better  knowledge  of 
requirements  with  closer  proximity  to  the 
actual  need,  the  Country  Store  can  initiate  or 
arrange  acquisition  of  products  which  meet 
the  user  requirements  and  command 
objectives.  Finally,  there  is  economies  of 
scale  in  having  the  volume  generated  by  a 
central  repository.  Opportunities  for  smaller 
“best-value”  or  cost-based  acquisitions  can 
be  easily  identified  and  accomplished  through 
the  Country  Store  structure,  with  a  quicker 
“turn-around”  from  initial  need  to  final 
satisfaction. 
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How  is  the  Country  Store  structured?  The 
Country  Store  is  a  “virtual”  store  made  up  of 
a  "wholesale”  operation  which  operates  the 
information  system  and  performs  central 
support  to  field  “retail”  stores  at  each 
Country  Store  base.  Each  AFMC  location 
and  the  Pentagon  has  a  “retail”  Country 
Store  operated  by  the  Base’s 
Communications  Group.  As  requirements 
are  identified  and  validated,  the  retail 
Country  Stores  use  the  information  system 
to  review  the  existing  contracts,  and  the 
listings  of  software  tools  available,  to  ensure 
that  a  duplicate  effort  is  not  undertaken. 

How  does  the  Country  Store  provide 
information?  The  Country  Store’s 
information  system  is  designed  to  provide 
access  to  all  levels  of  computer  capability 
and  expertise.  In  its  simplest  form,  the 
Country  Store  provides  automatic  reply  by  e- 
mail  of  any  or  all  of  the  Country  Store 
documents.  By  sending  a  message  to  the 
Country  Store,  a  general  catalog  is  provided 
which  is  an  index  of  all  of  the  Country  Store 
documents  and  resources.  To  receive  a 
document,  an  e-mail  listing  the  document 
name  in  the  message  field  provides  the 
document.  The  same  documents  are 
available  on-line  by  accessing  the  AFMC 
Bulletin  Board,  available  in  several  forms. 
Users  can  also  access  the  Country  Store 
directly  using  Gopher.  The  latest 
implementation  of  the  Country  Store  is  via 


the  World  Wide  Web.  The  Country  Store 
has  its  own  “home  page”  which  can  be 
accessed  using  a  Web  “browser”  such  as 
Netscape  or  Mosaic.  Under  the  Web 
implementation,  the  Country  Store  has  been 
able  to  link  directly  (via  hyperlink)  to  the 
actual  files  and  information  located  on  the 
Web,  without  having  to  duplicate  the 
information.  The  Web  constitutes  the  future 
of  the  Country  Store  as  users  gradually 
acquire  capability  to  access  by  this  method. 

The  Country  Store  story  is  only  beginning. 
The  opportunities  are  tremendous  to  achieve 
a  number  of  objectives.  For  example, 
electronic  commerce  is  being  looked  at  in 
depth,  as  a  way  of  achieving  even  greater 
benefits.  As  the  use  of  the  Country  Store 
grows,  even  with  the  present  structure, 
benefits  will  appear. 

In  summary,  the  AFMC  Country  Store 
provides  a  structure  and  centralized 
information  to  accomplish  a  number  of 
command-wide  objectives,  including 
implementation  of  best  practices,  increased 
compatibility  of  systems  through  standards 
and  technology  improvements,  closer 
attention  to  customer  needs  and  faster 
acquisition,  eliminating  unnecessary 
acquisitions,  cost  reduction,  achieving  higher 
productivity  with  less  people,  and  ultimately 
resulting  in  a  re-engineering  of  command 
efforts  to  better  support  our  mission. 
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ABSTRACT 

The  present  world  and  national  environment 
requires  substantial  changes  in  the  defense 
systems  acquisition  planning  and  manage¬ 
ment  process  including:  (1)  expansion  of  its 
purview  to  cover  the  life  cycle  of  defense 
systems,  not  only  acquisition,  and  (2)  inte¬ 
gration  of  Defense  Technology  Management 
(DTM)  with  life-cycle  Defense  Systems 
Management  (DSM).  To  support  this  thesis, 
we  discuss  a  life-cycle  concept  of  DSM,  and 
implications  of  the  new  Defense  Science  and 
Technology  Strategy  for  DTM.  We  propose 
"informational"  integration  of  DTM  with 
DSM  over  three  stages  of  a  defense  system's 
life  cycle  to  achieve  integrated  life-cycle  De¬ 
fense  Technology  and  Systems  Management 
(DT&SM).  Then  we  describe  applications 
of  new  Information  Management 
Technologies  (IMT)  that  will  enable  achiev¬ 
ing  "informational^"  integrated  DT&SM 
throughout  defense  system  life  cycles. 

To  advance  toward  this  goal,  we  propose 
four  initiatives  by  the  Under  Secretary  of 
Defense  for  Acquisition  and  Technology 
(USD(A&T))  to  begin  development  of  IMT- 
aided  tools  that  will  support  achievement  of 
integrated  DT&SM.  These  tools  will  enable 
any  DT&SM  professional:  (1)  to  locate  and 
ascertain  the  DTM  or  DSM  subject  areas  of 
every  available  major  organized  collection  of 
DT&SM  knowledge  and  information  (K&I) 
distributed  throughout  the  DT&SM  com¬ 
munity,  and  ultimately  (2)  to  access  a  chosen 
collection  and  retrieve  specific  K&I  relevant 
to  the  professional's  task  at  hand.  They 
would  also  help  Air  Force's  Integrated 
Weapon  System  Management  (IWSM) 
develop  a  truly  common  IWSM  language. 

INTRODUCTION 

The  post-Soviet  world  environment;  the 
prospect  of  limited  defense  budgets  through¬ 
out  the  foreseeable  future,  and  the  new  De¬ 
fense  Science  and  Technology  Strategy, 


combine  to  require  substantial  changes  in  the 
Defense  Systems  Acquisition  Management 
(DSAM)  process.  First,  they  require  expan¬ 
sion  of  its  purview  to  cover  every  stage  of  a 
defense  system's  life  cycle  (i.e.,  expansion  to 
a  life-cycle  DSM  process).  Secondly,  die 
new  DoD  strategy  for  Defense  Science  and 
Technology  Management  (shortened  in  this 
paper  to  Defense  Technology  Management 
or  DTM),  will  require  the  integration  of 
DTM  with  DSM  throughout  the  life  cycle  of 
every  defense  system.  This  cannot  be 
achieved  through  organizational  integration. 
Fortunately,  recent  advances  in  Information 
Management  Technologies  (IMT)  now  en¬ 
able  "informational"  integration  of  DTM  and 
life-cycle  DSM,  to  achieve  integrated  De¬ 
fense  Technology  and  Systems  Management 
(DT&SM)  throughout  each  defense  system's 
life  cycle. 

To  support  this  thesis,  this  paper  builds  upon 
concepts  first  outlined  in  our  research  papers 
for  previous  Acquisition  Research  Symposia 
(ARS):  1991,  "Achieving  Excellence  in  Man¬ 
agement  of  Defense  Systems"  1  and  1993, 
"Acquisition  Knowledge  &  Information 
Locator  System."  2  The  paper  also  considers 
important  new  developments  in  the  defense 
systems  management  (DSM)  environment, 
and  in  information  management  technologies 
(IMT),  to  update  and  extend  our  earlier  con¬ 
cepts  for  providing  every  DSM  professional 
more  timely  access  to  relevant  DSM 
knowledge  and  information  (K&I)  for 
systems  management  tasks  at  hand. 

One  new  development  concerns  a  kind  of 
K&I  that  DSM  professionals  must  now  con¬ 
sider  throughout  the  system's  life  cycle. 
Deputy  Secretaiy  of  Defense  John  Deutch 
announced  this  Administration's  new  Defense 
Science  and  Technology  Program  on  5  Oc¬ 
tober  1994,  and  two  new  implementing 
documents,  Defense  Science  and  Technology 
Strategy  and  Defense  Technology  Plan.3 
Thus,  Defense  Science  and  Technology 
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Management  became  a  new  integral  part  of 
the  three-stage  (acquisition,  operational 
employment  support,  and  final  disposition) 
life-cycle  management  process  that  we 
named  Defense  Systems  Management 
(DSM)  in  our  1991  ARS  paper. 

The  new  integrated  whole  body  of  DTM  and 
DSM  knowledge  will  be  represented  in  this 
paper  by  the  acronym  DT&SM  (Defense 
Technology  &  Systems  Management).  This 
integrated  DT&SM  knowledge  is  needed  to 
continually  manage  the  conception,  develop¬ 
ment  and  application  of  affordable  leading 
technology  throughout  the  life  cycle  of  all 
defense  systems.  Given  austere  defense 
budgets  for  the  foreseeable  future,  integrated 
DT&SM  knowledge  is  required  to  best 
assure  continued  technological  superiority  of 
our  defense  systems,  and  their  sustained 
operational  suitability  and  effectiveness  in  all 
stages  of  their  life  cycles. 

The  concepts  represented  by  the  acronyms 
DSM,  DTM  and  DT&SM  are  central  to 
improving  life  cycle  management  of  Defense 
systems.  Each  will  be  discussed  in  turn. 
These  acronyms  are  identified  here  so  we 
can,  at  the  beginning,  state  our  purpose  more 
precisely  and  provide  an  overall  framework 
which  will  enable  better  understanding  of 
these  concepts,  their  importance,  and  their 
interrelationships  when  they  are  discussed. 

Our  primary  purpose  is  to  propose  early 
development  of  two  defense  systems  man¬ 
agement  tools.  These  and  two  subsequent 
tools  will  enable  increasingly  integrated,  less 
costly,  and  more  effective  management  of 
defense  systems  to  assure  their  continued 
operational  effectiveness  over  their  life 
cycles.  Together,  these  tools  will  provide: 

a.  To  any  DT&SM  professional,  work¬ 
ing  on  any  aspect  of  a  defense  system's  life 
cycle, 

b.  Timely  access  to  specific  relevant 
DT&SM  knowledge  and  information  (K&I) 
that  is  available  in  any  K&I  SOURCE,  4 

c.  When  needed  by  die  professional  for 
a  DTM  or  DSM  task  at  hand. 

Proposed  Initiatives.  To  achieve  inte¬ 
grated  management  of  defense  technology 
and  systems  over  every  system's  life  cycle, 


we  propose  the  following  initiatives  by  the 
Under  Secretary  of  Defense  for  Acquisition 
and  Technology  (USD(A&T)): 

1.  Initiate  immediate  development  of  an 
on-line5  DoD  Macro-level  DT&SM 
Thesaurus  Structure  (of  the  terms 
representing  key  DT&SM  concepts,  and  the 
equivalence,  hierarchical  and  associative 
relationships  among  the  terms)  that  will 
enable  ultimate  development  of  a  two-level 
DoD  Thesaurus  of  the  Body  of  DT&SM 
Knowledge. 

2.  Then,  as  first  application  of  this  DoD 
macro-level  thesaurus  structure  of  key 
DT&SM  terms,  initiate  development  of  an 
on-line  DoD  Macro-level  DT&SM 
Knowledge  Locator  System  (based  on  a 
DoD  Thesaurus  of  Available  SOURCES  of 
DT&SM  K&I),  which  any  DT&SM  profes¬ 
sional  can  use  to  quickly  locate  available 
major  SOURCES  of  DT&SM  K&I. 
distributed  throughout  the  DT&SM 
community. 

With  this  Locator  System  alone,  any 
DT&SM  professional  with  a  DTM  or  DSM 
task  at  hand,  can  quickly  scan  all  available 
SOURCES  throughout  the  DT&SM  com¬ 
munity,  and  locate  and  select  those  with 
collections  most  likely  to  have  DTM/DSM 
K&I  that  are  pertinent  to  the  immediate  task. 
Then,  the  professional  can  use  the  particular 
micro-level  access  system,  provided  by  each 
selected  SOURCE  to  access  its  collection  of 
DTM/DSM  K&I,  to  retrieve  specific  K&I 
that  are  relevant  to  the  professional's  task  at 
hand. 

After  the  Locator  System  is  under  develop¬ 
ment,  two  other  applications  of  the  basic 
thesaurus  structure  tool  above,  will  assist 
further  integration  of  DT&SM.  The  first, 
prompted  by  the  next  initiative,  will  enable 
yet  more  direct,  timely,  selective  access  to 
specific  relevant  knowledge.  The  second, 
prompted  by  the  last  initiative,  will  promote  a 
common  DT&SM  language  for  communicat¬ 
ing  and  coordinating  effectively  in  life-cycle 
management  of  defense  systems. 

3.  Encourage  evolutionary  extension  of 
the  on-line  DoD  macro-level  DT&SM 
Thesaurus  Structure  down  into  every  major 
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SOURCE  of  DT&SM  K&I  that  has  been 
included  in  the  DoD  DT&SM  Knowledge 
Locator  System. 

Each  evolutionary  extension  down 
should,  with  USD(A&T)  coordination  and 
support,  be  developed  by  each  respective 
SOURCE.  The  same  IMT  and  thesaurus 
construction  aids  are  available  to  all. 
However,  each  organized  SOURCE  has  the 
best  working  knowledge  of  its  own  particular 
DT&SM  subject  matter,  and  which  terms 
would  be  most  useful  for  its  micro-level 
thesaurus  structure.  So,  each  SOURCE 
should  develop  its  own  on-line  micro-level 
thesaurus  of  the  special  DT&SM  K&I 
(e.g.,  documents,  data,  expert  consultation) 
that  are  available  in  the  SOURCE. 

An  example  of  a  major  SOURCE  is  the 
Manpower  and  Training  Research  Informa¬ 
tion  System  (MATRIS)  Office  of  the 
Defense  Technology  Information  Center 
(DTIC).  The  MATRIS  Office  collects, 
stores,  updates,  retrieves  and  disseminates 
information  on  people-related  research.  To 
increase  its  effectiveness  in  these  activities, 
the  MATRIS  Office  developed  the  MATRIS 
Indexing  and  Retrieval  Thesaurus  (MIRT).6 
This  MIRT  is  an  example  of  our  concept  of 
a  useful  micro-level  thesaurus  of  the 
DT&SM  K&I  contained  in  a  SOURCE'S 
major  organized  collection.  The  MIRT 
consists  of  a  hierarchical  list  of  indexing 
terms  with  definitions  for  the  indexing  terms, 
to  enable  consistent  accurate  classification  of 
all  K&I  items  indexed  in  the  MATRIS  data¬ 
base,  and  a  Keyword  Out  of  Context 
(KWOC)  to  aid  timely  selective  retrieval  of 
K&I  from  the  database. 

4.  Initiate  DoD  development  and  main¬ 
tenance  of  an  on-line  DoD  DT&SM 
Glossary  (and  subset  area  glossaries)  that 
provides  current  authenticated  definitions  for 
each  term  in  the  DT&SM  Thesaurus 
Structure,  or  an  operational  description  of 
the  concept  represented  by  each  term.  Each 
SOURCE  should  also  develop  a  companion 
glossary  for  the  on-line  micro-level  thesaurus 
that  it  develops,  and  each  SOURCE'S 
glossary  should  be  added  to  the  on-line  DoD 
DT&SM  Glossary.  These  glossaries  would 
serve  as  a  basis  for  an  increasingly  common 
defense  systems  management  language 
throughout  the  DT&SM  community. 


The  rest  of  our  paper  will  provide  back¬ 
ground  and  information  to  help  understand 
why  it  is  crucial  that  the  USD(A&T) 
immediately  initiate  development  of  the  DoD 
macro-level  thesaurus  structure  for  DT&SM, 
and  vigorously  support  the  three  applications 
of  the  structure  outlined  above. 

The  ultimate  objective  of  a  complete 
DT&SM  thesaurus  structure  (comprised  of 
both  the  DoD  macro-level  DT&SM 
thesaurus  structure  and  the  hundreds  of 
evolutionary  SOURCE-developed  micro- 
level  thesaurus  structures  extending  down 
from  the  DoD  macro-level  structure)  is  to 
provide  a  growing  universal  structure  of 
preferred  DT&SM  terms  for  the  DT&SM 
community.  DT&SM  professionals  in 
government,  industry  or  academia  can  then 
use  this  common  thesaurus  structure  and  its 
applications: 

a.  To  uniformly  classify,  organize,  and 
index  the  widely  distributed  body  of 
DT&SM  knowledge  and  information  (K&I) 
available  in  every  SOURCE'S  organized 
collections  of  DT&SM  K&I. 

b.  To  readily  locate  available  pertinent 
SOURCES,  when  needed;  and  then  to  access 
the  most  pertinent  SOURCES,  and  selectively 
retrieve  specific,  timely,  relevant  DT&SM 
K&I  for  coping  with  the  professional's  DTM 
or  DSM  task  at  hand. 

c.  To  make  progress  toward  a  truly 
common  DT&SM  language,  since  the 
DT&SM  indexing  terms  chosen  will  be  the 
terms  used  regularly  by  the  professionals  in 
managing  their  DTM  or  DSM  tasks.  This 
common  language  will  enable  DT&SM 
professionals  throughout  the  DT&SM  com¬ 
munity,  to  communicate  unambiguously  and 
with  understanding,  in  planning,  coordinating 
and  carrying  our  their  respective  DTM  and 
DSM  tasks  throughout  the  life  cycle  of  every 
defense  system. 

We  will  also  address  why  developing  and 
applying  this  DT&SM  thesaurus  structure 
can  enable  more  effective  use  of  the 
Information  Highway  to  achieve  integrated 
management  of  defense  technology  and  sys¬ 
tems  over  the  life  cycles  of  defense  systems. 
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INTEGRATION  CONCEPTS 

DSM  Concept  In  our  paper  four  years 
ago,  "Achieving  Excellence  in  Management 
of  Defense  Systems,”  we  identified  the  need 
for  and  benefits  of  extending  the  domain  of 
Defense  Systems  Management  (DSM)7  to 
cover  all  three  stages  of  the  life  cycle  of  each 
defense  system — not  only  its  acquisition,  but 
also  its  operational  employment  support 
(including  field  updates  and  modifications) 
and  its  final  disposition.  We  recognized  that 
the  boundary  barriers  between  each  stage  are 
the  result  of  significant  differences  in  the 
"culture,”  language  and  primary  objectives  of 
the  professionals  working  in  each  stage. 
These  differences  have  caused  serious  dis¬ 
connects  and  misunderstandings,  and  waste¬ 
ful  discontinuities  in  the  system  management 
processes  when  each  defense  system  is  trans¬ 
ferred  to  the  next  stage  in  its  life  cycle. 
These  differences  have  impeded  feedback  of 
lessons-leamed  knowledge  and  operational 
information  from  later  stages,  which  could 
improve  substantially  the  conception,  design 
and  production  of  new  systems.  These  differ¬ 
ences  have  also  impeded  "feed  forward"  of 
new  technology  K&I  to  enable  more  timely 
cost-saving  modification  and  update  of  cur¬ 
rent  operational  systems  to  enhance  their 
technological  superiority  and  continued 
operational  suitability  and  effectiveness. 
These  differences  still  impede  multidiscipli¬ 
nary  concurrent  engineering  aimed  at  the 
simultaneous  integration  of  all  life  cycle 
elements,  such  as  producibility,  reliability, 
and  maintainability,  into  the  systems  design 
process. 

Our  analysis  showed  that  extending  the 
domain  of  defense  systems  management 
throughout  the  entire  life  of  each  defense 
system  would  give  significant  payoffs — in 
enhanced  operational  suitability  and  opera¬ 
tional  effectiveness  of  each  defense  system, 
and  in  dollars  saved,  more  productive  de¬ 
fense  systems  management  practices,  and 
improved  systems  management  education. 

We  used  "systems  thinking"  to  define  and 
discuss  the  Defense  Systems  Management 
(DSM)  domain  that  included  and  extended 
beyond  the  acquisition  stage  to  cover  each 
system's  life  cycle.  We  discussed  the 


boundaries  of  the  DSM  domain,  and  how 
they  spanned  both  a  great  national  security 
managerial  hierarchy,  and  a  long  DSM  time 
line  from  initial  conception  to  final  disposi¬ 
tion  of  each  system.  The  managerial  hierar¬ 
chy  ranges  from  establishing  national  defense 
policies  that  involve  defense  systems,  down 
through  numerous  DoD  and  other  govern¬ 
ment  organizational  hierarchies  to  the  man¬ 
agement  level(s)  where  these  policies  are 
finally  executed  for  each  system. 

The  top  part  of  Figure  1  is  a  1993  updated 
portrayal  of  our  1991  concept  of  Defense 
Systems  Management  (DSM).  Its  upper  part 
portrays  our  view  of  management  over  a 
defense  system's  life  cycle,  with  three  con¬ 
secutive  management  stages:  acquisition, 
operational  employment  support,  and  final 
disposition.  It  also  portrays  K&I  Process 
Systems  that  are  required  to  support  the 
managers  in  each  stage.  Three  process  sys¬ 
tems  which  cross  stage  boundaries  and  need 
K&I  feedback  and  "feed  forward"  across  the 
boundaries  are  identified.  They  concern  sys¬ 
tem  test  and  evaluation  (DT&E  and  OT&E), 
man-machine  integration  management  (M- 
MIM)  to  achieve  most  effective  performance 
of  the  total  operational  system,  and  logistics 
and  infrastructure  support  management 
(L&ISM).  There  are  many  other  K&I  proc¬ 
ess  systems.  Many  already  exist,  others 
should  be  developed,  and  all  need  to  be 
integrated.  This  portrayal  of  DSM  also 
identifies  some  of  die  many  kinds  of  DSM 
knowledge  and  information  in  various  major 
organized  collections  of  SOURCES  that  a 
DoD  Macro-level  DT&SM  Knowledge 
Locator  System  could  help  locate  and  access. 

The  boundaries  between  DSM  stages  are  the 
result  of  some  significant  differences  in  the 
objectives  and  "culture,"  and  thus  differences 
in  the  language  (terms  used  in  communica¬ 
tion)  by  die  professionals  working  in  each 
stage.  Disconnects  and  misunderstandings  in 
communications,  and  discontinuities  in  man¬ 
agement  processes,  result  when  professionals 
working  in  a  particular  process  that  crosses  a 
stage  boundary  (e.g.,  the  L&ISM — Logistics 
and  Infrastructure  Management  process) 
don't  have  a  completely  common  language 
(i.e.,  common  concepts,  with  each  concept 
represented  by  its  own  commonly  understood 
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term).  Integrating 
the  DSM  stages  will 
be  difficult,  if  not 
impossible,  without 
common  concepts 
and  a  common 
working  language 
for  all  stages  of  each 
system's  life  cycle. 

The  recent  new  re¬ 
quirement  outlined 
next,  to  integrate 
DTM  with  every 
stage  of  DSM 
greatly  increases  the 
urgency  of  achieving 
common  concepts 
and  a  truly  common 
language  for  inte¬ 
grated  Defense 
Technology  and 
Systems  Manage¬ 
ment  (DT&SM). 

Figure  1.  Integrated  DEFENSE  TECHNOLOGY  &  SYSTEMS  MANAGEMENT  (DT&SM) 
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DTM  Concept. 

Defense  Technology  Management  (DTM)  is 
an  evolutionary  new  concept.  It  began 
evolving  in  the  post  cold-war  environment 
from  the  traditional  program  of  Research, 
and  Exploratory  and  Advanced  Develop¬ 
ment.  For  many  years  this  R&D  program, 
although  not  formally  recognized  as  part  of 
defense  systems  acquisition  management,  has 
provided  most  of  the  technology  needed  in 
acquisition  of  new  defense  systems.  Now, 
shrinking  Defense  budgets,  prospects  of 
technological  surprise,  and  emergence  of 
many  different  and  unforeseen  threats  in  the 
post-Soviet  world,  have  expedited  evolution 
of  this  new  concept  that  DTM  is  an  integral 
part  of  life-cycle  defense  systems  manage¬ 
ment.  The  importance  of  the  new  DTM 
concept  is  evidenced  by  the  recent  addition 
of  Technology  in  the  title  of  Under  Secretary 
of  Defense  for  Acquisition  (USD  A),  now  the 
short  title  is  USD(A&T). 

On  5  October  1994,  the  Deputy  Secretary  of 
Defense  and  the  Director  of  Defense  Re¬ 
search  and  Engineering  (DDR&E),  an¬ 
nounced  this  Administration's  new  Defense 
Science  and  Technology  (S&T)  Program, 
and  the  new  Defense  Science  and  Technol¬ 


ogy  Strategy  and  Defense  Technology  Plan  3 
for  implementing  it.  The  vision  of  this  S&T 
program  is  to  "Develop  and  transition  supe¬ 
rior  technology  to  enable  affordable,  decisive 
military  capability  and  to  enhance  economic 
security."  The  program  intends  to  improve 
productivity,  reduce  costs,  and  above  all 
assure  technological  superiority. 

The  aspects  of  science  and  technology  that 
the  DoD  pursues  in  the  S&T  program,  and 
with  what  priority,  will  be  determined  by  our 
military  needs,  recognizing  that  the  program's 
content  will  be  constrained  by  limited 
budgets.  Five  Future  Joint  Warfighting 
Capabilities  have  been  identified  by  the  Joint 
Staff  and  the  Joint  Requirements  Oversight 
Council  as  most  needed  by  the  U.S. 
Combatant  Commands. 

Two  strategic  investment  priorities,  both  new 
to  the  Defense  S&T  program,  will  maximize 
the  extent  that  the  S&T  vision  can  be 
achieved  within  budget  constraints.  One, 
with  specific  exceptions  for  some  critical 
defense-unique  technologies  such  as  nuclear 
weapons  and  missile  guidance,  is  the  priority 
for  dual-use  technology  development.  This 
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contributes  to  building  a  common  commer¬ 
cial  and  defense  industrial  base,  which  will 
serve  most  defense  needs  better  and  also 
enhance  U.S.  economic  competitiveness. 

The  second  is  to  consider  affordability  as  an 
integral  part  of  the  S&T  program.  In  the 
past,  the  threat  demanded  and  the  budget 
permitted  the  S&T  program  to  focus  on 
increased  performance  along  all  warfighting 
dimensions,  e.g.,  fighter  aircraft  speed,  cold 
weather  endurance,  radiation  hardness. 
Today,  materiel  and  systems  must  be  devel¬ 
oped  at  lower  cost,  be  longer-lived,  and  be 
incrementally  enhanced  through  planned 
upgrades.  Based  on  analysis  of  the  capabili¬ 
ties  that  the  Combatant  Commands  need 
most  and  the  technological  opportunities  that 
exist  today,  priority  will  be  placed  on  three 
technology  areas:  information  science  and 
technology,  modeling  and  simulation,  and 
sensors. 

The  S&T  program  is  traditionally  described 
as  having  three  separate  and  identifiable 
elements:  Basic  Research,  Exploratory 
Development,  and  Advanced  Development. 
It  functions,  however,  as  a  continuum  to 
advance  technology,  with  the  Advanced 
Technology  Development  program  structured 
to  apply  technological  advances  to  provide 
military  capability.  In  the  Defense 
Technology  Plan,  technology  efforts  are 
structured  into  nineteen  areas.  This  plan, 
updated  annually,  will  be  used  to  manage 
development  and  application  of  new  technol¬ 
ogy  to  provide  adequate  defense  capability. 

As  indicated  earlier,  the  vision  of  the  S&T 
program  is  to  improve  productivity  and  re¬ 
duce  costs,  and  above  all  assure  technological 
superiority.  Achieving  this  involves 
managing  technology  development  and  appli¬ 
cation  to  assure  technologically  superior 
defense  capabilities.  In  our  paper  we  call  this 
Defense  Technology  Management  (DTM). 

In  Figure  1,  we  have  portrayed  DTM  and 
Defense  Systems  Management  (DSM)  as 
integral  parts  of  integrated  Defense 
Technology  and  Systems  Management 
(DT&SM).  As  an  integral  part  of  DT&SM, 
DTM  begins  before  fire  acquisition  stage, 
before  the  start  of  any  defense  system's  life 


cycle,  and  seeks  to  assure  the  technological 
superiority  of  new  system  acquisitions.  DTM 
continues  interacting  with  DSM  during  each 
DSM  stage  to  assure  continued  military 
superiority,  and  sustained  operational 
suitability  and  effectiveness  of  each  defense 
system  throughout  its  life  cycle. 

The  objective  of  the  underlying  Defense 
Basic  Research  program  in  DTM  is  to  pro¬ 
duce  knowledge  in  a  science  or  engineering 
area  that  is  militarily  relevant.  It  is  a  long 
term  investment  with  emphasis  on  opportuni¬ 
ties  far  into  the  future.  The  current  pro¬ 
gram's  investments  are  in  twelve  scientific 
and  engineering  areas  listed  in  Defense 
Science  and  Technology  Strategy.  The  pro¬ 
gram  intends  to  be  a  source  of  new  knowl¬ 
edge  that  may  be  recognized  as  militarily 
relevant  at  some  point  in  the  DSM  life  cycle. 

Exploratory  Development  and  Advanced 
Development  mature  technologies.  Explora¬ 
tory  Development,  which  may  include  proto¬ 
types  embodying  a  technology,  takes  research 
results  and  provides  proof  of  concept 
experiments  and  evaluations  built  around 
models  and  laboratoiy  experiments.  The 
Advanced  Technology  Development  pro¬ 
gram  is  structured  to  apply  technology 
advances  to  provide  military  capability. 
Current  technology  efforts  are  structured  in 
the  Defense  Technology  Plan  into  nineteen 
technology  areas  for  oversight  purposes. 
Specific  results  of  these  experiments  and 
technology  efforts  also  may  be  found 
applicable  and  put  to  use  at  any  point  in  the 
DSM  life  cycle. 

Advanced  Concept  Technology  Demonstra¬ 
tions  (ACTDs)  are  a  new  aspect  of  the  S&T 
program.  They  are  the  focused  successors  of 
the  seven  broad  S&T  thrusts  pursued  over 
the  past  few  years.  ACTDs  are  tightly  fo¬ 
cused  on  specific  military  concepts.  They  are 
user-oriented  with  three  motivations:  (1)  to 
have  the  user  gain  an  understanding  of  an 
ACTD's  military  utility,  and  to  evaluate  it 
before  committing  to  acquisition;  (2)  to 
develop  corresponding  concepts  of  operation 
and  doctrine  that  make  best  use  of  the  new 
capability;  and  (3)  to  provide  residual  opera¬ 
tional  capability  to  the  forces.  The  outcome 
of  an  ACTD,  and  whether  or  not  an  ACTD 
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initiates  acquisition  as  a  new  system,  or  as  a 
technological  upgrade  of  an  operational  sys¬ 
tem,  is  judged  by  its  operational  users. 
ACTDs  are  a  focused  means  of  integrating 
DTM  into  DSM  over  a  system's  life  cycle. 

Integrated  DT&SM  Concept.  The 

vision  is  that  DTM  and  DSM  will  become 
truly  integral  parts  of  the  whole  integrated 
process  for  managing  defense  systems 
throughout  their  life  cycles — integrated 
DT&SM  (Defense  Technology  and  Systems 
Management).  The  problem  is  how  to 
accomplish  an  integrated  DT&SM  and  best 
assure  real  operational  integration  of  all  DTM 
and  DSM  K&I  in  order  to  achieve 
affordable,  technologically  superior,  and 
environmentally  safe  defense  systems 
throughout  every  stage  of  their  life  cycles. 

Figure  1  portrays  four  different  major  proc¬ 
esses  that  must  be  performed,  first  in  manag¬ 
ing  defense  systems  over  three  stages  of  their 
life  cycles  (DSM),  and  then  as  of  October 
1994,  concurrently  managing  technology 
(DTM)  and  integrating  it  into  each  stage. 
Integrating  DTM  into  DSM  aims  to  reduce 
costs  of  the  systems  in  each  stage,  while  also 
assuring  their  technological  superiority 
throughout  their  life  cycles  in  the  changing 
and  uncertain  post-Soviet  environment. 

Long  standing  organizational  attempts  to 
overcome  the  barriers  between  these  major 
processes  have  been  largely  unsuccessful. 
However,  a  new  paradigm  is  emerging.  The 
Acquisition  Management  Integrated  Weapon 
System  Management  (IWSM)  Guide, 8  1  Oc¬ 
tober  1993,  describes  how  the  Air  Force 
Materiel  Command  (AFMC)  is  addressing 
overcoming  these  barriers  with  its  new 
(IWSM  White  Paper,  January  1992)  and  still 
evolving  IWSM  concept.  The  concept  is 
designed  to  provide  single  product  manage¬ 
ment  for  all  aspects  of  a  system  throughout 
its  life  cycle — from  cradle  to  grave.  One  key 
IWSM  principle  is  Integrated  Product  Teams 
(IPT)  with  their  cross-functional  representa¬ 
tion  and  product  emphasis  to  achieve  an 
effective  total  systems  approach.  We  believe 
that  IWSM  represents  a  great  step  in  over¬ 
coming  barriers  between  the  four  different 
major  processes.  It  should  be  emulated 
throughout  the  DT&SM  community.  The 


IWSM  concept  of  Integrated  Product  Teams 
(IPT)  seems  to  fit  David  Packard's  criteria  for 
centers  of  management  excellence ,9 

However,  in  implementing  IWSM,  AFMC  is 
encountering,  a  real  barrier  to  integration 
— lack  of  a  common  language  among  all 
functions  involved.  And  to  be  effective,  the 
IWSM  Integrated  Product  Teams  still  face 
the  universal  problem  of  getting  timely 
knowledge  and  information  relevant  to  tasks 
at  hand.  Implementing  our  proposed 
initiatives  should  help  IWSM  programs  cope 
better  with  both  problems. 

We  believe  that  truly  integrated  DT&SM  are 
necessary  for  cultivation  of  the  centers  of 
management  excellence  which  Chairman 
David  Packard,  in  his  Foreword  to  the  1986 
Packard  Report,  stated  is  required  in  the  large 
complex  enterprise  of  national  defense. 

Integrated  DT&SM  can  (a)  provide  the 
integrated  DSM  and  DTM  K&I  required  to 
cultivate  centers  of  management  excellence 
throughout  the  DT&SM  community,  and  (b) 
provide  to  the  professionals  in  each  center, 
timely  and  selective  on-demand  access  to 
relevant  DSM  and/or  DTM  K&I  that  each 
professional  requires  for  five  main  purposes: 

1.  To  continually  know  and  have  a 
common  understanding  of  the  goals  and 
objectives  of  DoD's  centralized  policies. 

2.  To  clearly  coordinate  each  center's 
respective  efforts  in  its  area  of  responsibili¬ 
ties,  toward  coordinated  execution  of  the 
commonly  understood  centralized  defense 
systems  management  policies,  in  order  for  all 
to  contribute  productively  to  achieving  DoD's 
goals  and  objectives  for  assuring  our  national 
defense. 

3.  To  develop  and  maintain  job-  and 
task-related  specialized  know-how. 

4.  To  explore  new  and  better  ways  for 
getting  each  professional's  job  done. 

5.  To  perform  with  excellence  any 
DSM  or  DTM  management  task  at  hand. 

Truly  integrated  DT&SM  requires  and 
depends  upon  IMT-enabled  "informational'' 
integration  of  all  DSM  and  DTM  K&I 
required  by  DT&SM  professionals  for  their 
DSM  or  DTM  management  tasks  in  any 
aspect  of  a  defense  system's  life  cycle.  No 
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other  practical  alternative  is  available. 
Changes  in  organizational  structure  and 
processes  may  correct  some  authority- 
responsibility  deficiencies  (while  creating 
others).  However,  they  do  not  solve  the 
problems  of  assuring  timely  access,  by 
DT&SM  professionals  working  in  any  part  of 
a  defense  system's  life  cycle,  to  selected 
DSM  and/or  DIM  K&I  that  are  relevant  for 
a  systems  management  task  at  hand. 

In  the  complex  bureaucracy  concerned  with 
national  defense,  integrated  DT&SM  cannot 
be  achieved  organizationally.  But,  it  can  be 
achieved  operationally— by  applying  new 
information  management  technologies  (EMT) 
to  achieve  "informational”  integration  of 
DT&SM  K&I. 

The  need  for  integrated  DT&SM  K&I  is 
critical,  given  the  limited  defense  budget 
outlook  and  the  present  world  environment. 
We  see  no  alternative  to  IMT-enabled 
"informational"  integration  of  DT&SM  K&I 
that  are  widely  distributed  throughout  the 
DT&SM  community  (government,  industry 
and  academia).  Considering  both  we  identify 
some  IMT  elements  that  should  be 
considered  in  progressing  toward  integrated 
DT&SM. 

IMT  TOOLS 

Information  Highway.  Much  of  the 
information  highway  that  already  exists  can 
be  used  by  professionals  throughout  the 
DT&SM  community:  to  quickly  obtain  for 
management  tasks  at  hand,  relevant  DTM 
and  DSM  K&I  from  pertinent  widely  distrib¬ 
uted  organized  collections  of  SOURCES,  and 
to  coordinate  their  systems  management 
actions. 

What  these  professionals  need  now  are:  (a) 
better  DT&SM  on-  and  off-ramps  to  connect 
each  professional  to  all  organized  collections 
of  DTM  and  DSM  K&I  that  are  maintained 
by  SOURCEs;  (b)  better  guidebooks  for  first 
locating  pertinent  SOURCEs;  and  then,  for 
selectively  accessing  the  organized  collections 
of  pertinent  SOURCEs,  and  retrieving  K&I 
that  are  relevant  for  a  task  at  hand;  and  (c)  a 
common  DT&SM  language  for  communicat¬ 
ing  with  understanding  in  coordinating  their 


management  actions  to  assure  technologically 
superior,  and  operationally  suitable  and 
effective  defense  systems  throughout  each 
system's  life  cycle. 

Like  any  useful  highway,  a  key  factor  that 
makes  the  information  highway  useful  is  the 
standards  that  are  applicable  to  its  component 
elements.  As  part  of  the  U.S.  National  In¬ 
formation  Infrastructure,  the  Federal  gov¬ 
ernment  has  established  the  Government 
Information  Locator  Service  (GILS)  Stan¬ 
dard  to  help  the  public  locate  and  access 
government  information  (e.g.,  economic 
data,  environmental  data,  technical  informa¬ 
tion).  OMB  Bulletin  No.  95-01,  dated 
December  7,  1994,  established  the  GILS. 
The  National  Institute  of  Standards  and 
Technology  (NIST)  issued  Federal  Informa¬ 
tion  Processing  Publication  (FIPS  Pub)  192, 
dated  December  7,  1994,  to  announce  the 
standard  for  the  Application  Profile  for  the 
GILS— a  voluntary  international  standard 
with  mandatory  application  for  Federal 
agencies  establishing  locators  for  information. 
This  standard  must  be  considered  in  the 
proposed  development  of  the  DoD  Macro¬ 
level  DT&SM  Knowledge  Locator  System. 

Many  who  are  users,  or  who  are  at  least 
aware  of  Internet,  recognize  that  Internet  is 
an  integral  part  of  today's  information  high¬ 
way,  and  that  it  is  completely  open  to  the 
public  world-wide.  They  may  be  wary  about 
developing  better  DT&SM  on-  and  off¬ 
ramps,  and  better  guidebooks  for  accessing 
DT&SM  K&I,  because  they  fear  adverse 
effects  from  using  these  tools  on  Internet  and 
on  other  aspects  of  the  information  highway 
that  are  completely  open  to  the  public.  This 
fear  about  using  IMT  to  develop  and  use 
these  new  systems  management  tools  is  not 
warranted. 

The  U.S.  intelligence  community  has  created 
its  own  Internet,  dubbed  Intelink,10  which  is 
based  on  the  same  technology  used  to  run 
and  navigate  the  original  Internet.  This  new 
Intelink  network  for  sharing  supeisecret 
information,  officially  began  operating  in  late 
1994.  When  the  bugs  are  worked  out  and 
the  final  tystem  is  in  place,  it  will  allow  ana¬ 
lysts,  policymakers,  and  workers  in  the  field 
who  have  Top  Secret  security  clearances,  to 
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tap  quickly  and  directly  from  their  own 
computers  into  classified  information  in 
various  parts  of  the  national  security 
bureaucracy. 

We  do  not  expect  Intelink  to  be  available  for 
defense  systems  management.  However, 
much  of  the  technology  used  to  run  and 
navigate  Intelink  can  be  used  to  quickly 
develop  a  DT&SMlink  network,  when  the 
DT&SM  on-  and  off-ramps,  the  DT&SM 
guidebooks  and  a  common  DT&SM  lan¬ 
guage  are  developed  and  ready  to  be  used  on 
a  DT&SMlink  network  to  manage  defense 
systems  throughout  their  life  cycles. 

Thesaurus  Structure.  Developing  a  DoD 
Macro-level  DT&SM  Thesaurus  Structure  is 
prerequisite  to  all  other  initiatives  we  pro¬ 
posed  at  the  beginning — initiatives  for  devel¬ 
oping  tools  that  can  enable  increasingly 
integrated,  less  costly,  and  more  effective 
management  of  defense  systems  to  assure 
their  continued  operational  effectiveness  over 
their  life  cycles. 

This  DT&SM  thesaurus  structure  is  essential: 
to  the  development  of  DT&SM  on-  and  off¬ 
ramps  to  the  information  highway,  to  devel¬ 
opment  of  guidebooks  for  accessing  and 
retrieving  relevant  K&I,  to  development  of  a 
common  DT&SM  language  for  communicat¬ 
ing  and  managing  defense  systems  more 
effectively,  and  will  have  other  uses  in  the 
integration  of  DT&SM  K&I.  The  DT&SM 
thesaurus  structure  is  key  to  "informational" 
integration  of  DT&SM  K&I. 

The  thesaurus  structure,  in  the  context  of 
this  paper,  is  a  vocabulary  of  controlled 
indexing  language.  The  structure  will  also  be 
used  as  a  retrieval  language.  It  comprises  the 
terms  used  by  DT&SM  professionals  in 
communicating  and  coordinating  their  re¬ 
spective  efforts  in  performing  two  major 
types  of  management  tasks  to  achieve 
DT&SM  objectives: 

•  DSM  tasks  in  managing  defense  sys¬ 
tems  throughout  their  life  cycles,  including 
their  acquisition,  test  and  evaluation,  man- 
machine  integration,  upgrade,  operational 
training,  logistical  and  infrastructure  support, 
and  final  disposition. 


•  DTM  tasks  in  managing  militarily 
relevant  science  research  and  technology 
development,  to  achieve  new  technological 
advances  that  may  be  required,  in  the  acqui¬ 
sition  of  a  new  defense  system,  or  in  a  sys¬ 
tem's  upgrade  or  field  modification  during 
any  stage  of  its  life  cycle,  to  assure  its 
affordability,  technological  superiority, 
environmental  safety,  and  operational 
effectiveness. 

The  best  IMT  aid  we  have  found  for  plan¬ 
ning  development  of  the  DoD  Macro-level 
DT&SM  Thesaurus  Structure  is  the  book, 
Thesaurus  Construction ,n  published  in  1987 
by  The  Association  for  Information 
Management  in  London.  Among  items  a 
useful  thesaurus  structure  should  include,  it 
discusses  the  following  three  types  of  rela¬ 
tionships  among  indexing  terms.12  The  rela¬ 
tionships  are  defined  by  ISO  Standard  2788. 

•  Equivalence— "the  relationship  be¬ 
tween  preferred  and  non-preferred  terms 
where  two  or  more  terms  are  regarded,  for 
indexing  purposes,  as  referring  to  the  same 
concept."  The  preferred  term  is  the  one  cho¬ 
sen  to  represent  the  concept  for  indexing. 
The  non-preferred  terms  form  an  entry 
vocabulary  directing  the  user  to  the  preferred 
term.  The  following  conventions  are  used  to 
express  the  reciprocal  relationship:  UF  (use 
for)  written  as  a  prefix  to  the  non-preferred 
term;  USE  written  as  a  prefix  to  the  preferred 
term. 

•  Hierarchical — shows  "levels  of 
superordination  and  subordination.  The 
superordinate  term  represents  a  class  or 
whole,  and  the  subordinate  terms  refer  to  its 
members  or  parts."  This  relationship  is  a 
basic  feature  of  a  thesaurus,  distinguishing  it 
from  unstructured  term  lists.  The  following 
conventions  are  used  to  express  the  recipro¬ 
cal  relationship:  BT  (broader  term)  written  as 
a  prefix  to  the  superordinate  term;  NT 
(narrower  term)  written  as  prefix  to  the 
subordinate  term. 

•  Associative — shows  terms  that  are 
related,  but  in  a  way  other  than  described  in 
the  two  previous  relationships  (e.g.,  two  quite 
different  processes  may  be  associated  by  both 
being  required  to  complete  a  task).  The  re¬ 
ciprocal  relationship  is  distinguished  by  the 
abbreviation  RT  (related  term)  for  associated 
terms. 
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The  DT&SM  two-level  thesaurus  structure 

will  comprise: 

•  One  DoD  macro-level  DT&SM 
thesaurus  structure — to  be  used  first  to  de¬ 
velop  a  DoD  Thesaurus  of  Available 
SOURCES  of  DT&SM  K&I — that  can  help 
locate  all  available  organized  special  collec¬ 
tions  of  K&I,  distributed  widely  throughout 
the  DT&SM  community.  (See  Locator 
System  in  next  section.) 

•  Numerous  micro-level  thesaurus 
structures — each  developed  by  the  SOURCE 
of  an  available  organized  collection  of 
DT&SM  K&I — which  will  become  evolu¬ 
tionary  extensions  down  from  lowest  hierar¬ 
chical-level  terms  of  the  DoD  macro-level 
DT&SM  thesaurus  structure. 

As  explained  above  in  the  third  proposed 
initiative,  each  SOURCE  should  develop  its 
own  micro-level  thesaurus  structure  and  a 
thesaurus  of  the  special  DT&SM  K&I  that 
are  available  from  the  SOURCE'S  organized 
collection. 

The  MATRIS  Indexing  and  Retrieval- 
Thesaurus  (MIRT)  6  is  described  earlier  un¬ 
der  the  third  proposed  initiative,  as  an  exam¬ 
ple  of  our  concept  of  a  useful  structured 
micro-level  thesaurus  of  the  DT&SM  K&I 
available  from  a  K&I  SOURCE.  The 
MATRIS  Office  developed  its  MIRT  to  im¬ 
prove  indexing  and  retrieval  of  the  K&I  that 
are  available  in.  the  MATRIS  data¬ 
base — manpower  and  personnel,  education 
and  training,  simulation  and  training  devices, 
and  human  factors  engineering  K&I.  The 
MIRT  includes  three  presentations  of  its 
thesaurus  structure: 

•  MIRT  Hierarchical  List  of  Indexing 

Terms 

•  MIRT  Definitions  of  Indexing  Terms 

•  MIRT  Keyword  Out  of  Context 

(KWOC) 

These  MIRT  presentations  will  be  maintained 
and  used  by  the  MATRIS  Office  to  collect, 
store,  update,  retrieve  and  disseminate 
information  on  people-related  research  in  its 

database. 

No  example  exists  of  the  proposed.  DoD 
macro-level  DT&SM  thesaurus  structure. 
Nor  cat  one  be  constructed  from  the  bottom 
up  (i.e.,  from  the  micro-level  thesauri  of  K&I 


available  in  the  organized  colections  main¬ 
tained  by  all  SOURCEs),  since  the  location 
of  all  SOURCEs  is  not  yet  known,  and  many 
SOURCES  have  not  developed  a  structured 
thesaurus.  The  DoD  macro-level  DT&SM 
thesaurus  structure  must  be  constructed  from 
top  down. 

In  1991  we  proposed  development  of  a 
Defense  Systems  Acquisition  Management 
(DSAM)  Taxonomy-Glossary.  The  purpose 
of  the  DSAM  Taxonomy-Glossary  was  to 
provide  the  structure  for  a  DSAM  Structure 
&  Information  System  (DSAM-SIS).13  We 
proposed  developing  the  DSAM-SIS  as  a 
useful  tool  for  DSAM  professionals.  Those 
DSAM  professionals  in  charge  of  organized 
DSAM  collections  could  use  its  Taxonomy- 
Glossary  Structure  to  classify  and  index  the 
K&I  in  their  respective  DSAM  collections. 
Then,  all  DSAM  professionals  could  use  its 
Information  System  to  access  any  of  these 
widely  distributed  Taxonomy-indexed 
organized  DSAM  coEections,  and  selectively 
retrieve  relevant  DSAM  K&I  for  a  task  at 
hand.  Once  the  usefulness  of  DSAM-SIS 
was  demonstrated  in  acquisition,  we  believed 
it  would  be  extended  to  cover  the  other  DSM 
life-cycle  stages. 

The  completely  revised  DODD  5000.1, 
"Defense  Acquisition,"  and  DODI  5000.2, 
"Defense  Acquisition  Management  Policies 
and  Procedures,"  were  issued,  February  23, 
1991.  We  began  planning  to  use  digitized 
versions  to  construct  a  DSAM  Taxonomy- 
Glossary  (basically  a  DSAM  thesaurus 
structure).  The  terms  and  associated 
definitions  in  DODD  5000.1  and  DODI 
5000.2  were  terms  that  DSAM  professionals 
use  in  communicating  and  coordinating  their 
DSAM  tasks  and  acquisition  efforts.  So  a 
useful  DSAM  Taxonomy-Glossary  uses 
these  terms  and  definitions  as  the  vocabulary 
of  its  controlled  indexing  language. 

Using  Thesaurus  Construction  11  as  the 
guide  to  construct  the  DSAM  Taxonomy- 
Glossary  would  require  a  large  manual 
effort.  It  was  written  in  1987  before  great 
advances  in  personal  computer  software. 
Since  1991  we  have  been  searching  widely, 
even  at  the  Library  of  Congress,  for  informa¬ 
tion  about  computer  utilities  specifically 


designed  to  facilitate  thesaurus  construction. 
We  recently  queried  the  publishers  of 
Thesaurus  Construction.  They  told  us  that 
the  1987  edition  is  still  the  current  edition, 
but  gave  us  a  list  of  four  PC  software  pack¬ 
ages  which  aid  construction  of  thesauri.  We 
obtained  a  demonstration  copy  of  one,  Term 
Manager:  Structured  Thesaurus  System1* 
for  constructing  and  maintaining  thesauri. 
We  planned  to  use  Thesaurus  Construction 
and  the  demo  version  of  Term  Manager, 
with  a  digitized  version  of  DODD  5000.1 
and  DODI  5000.2  as  sources  of  indexing 
terms  and  definitions,  to  build  a  small  demo 
DSM  macro-level  thesaurus-structure  model. 

When  we  obtained  a  hard  copy  of  M1RT  in 
November  1994,  we  considered  trying  fur¬ 
ther  to  design  our  DSM  macro-level 
thesaurus-structure  model,  such  that  MIRT 
could  be  viewed  as  a  micro-level  thesaurus 
structure  that  extended  down  from  our 
demonstration  DSM  macro-level  thesaurus 
structure.  However,  late  receipt  of  both 
Term  Manager  and  MIRT,  coupled  with  an 
unrelated  emergency  two  months  before  due 
date,  prevented  developing  a  demonstration 
DSM  macro-level  thesaurus  structure  in  time 
for  this  paper. 

Nevertheless,  we  strongly  recommend  that 
the  Under  Secretary  of  Defense  for  Acquisi¬ 
tion  and  Technology  (USD(A&T))  initiate 
development  of  the  DoD  Macro-level 
DT&SM  Thesaurus  Structure,  considering 
all  SOURCES  of  available  organized  collec¬ 
tions  of  DT&SM  K&I  that  are  widely  dis¬ 
tributed  throughout  the  DT&SM  community, 
in  government,  industry  and  academia.  This 
thesaurus  structure  is  the  key  management 
tool  for  integrating  DT&SM,  beginning  with 
its  use  and  application  in  the  other  three 
initiatives  proposed  in  the  Introduction. 

Knowledge  Locator  System.  The  body 
of  DT&SM  knowledge  and  information 
(K&I)  is  enormous,  and  widely  distributed  in 
thousands  of  different  organized  collections 
of  particular  categories  of  DT&SM  K&I 
(i.e.,  SOURCEs),  throughout  the  DT&SM 
community — throughout  the  DOD  and  other 
government  organizations,  industry  and 
academia.  As  referenced  in  the  Introduction, 
the  term  K&I  SOURCE,  or  SOURCE,  is 


used  in  this  paper  to  indicate  any  organized 
collection  of  DT&SM  K&I  (e.g.,  a  special 
library,  repository,  archive,  database,  center 
of  expertise,  information  analysis  center)  that 
is  available  for  access  by  DT&SM 
professionals  to  retrieve  specific  K&I  that  are 
relevant  to  a  task  at  hand.  These  are 
collections  that: 

•  Concern  different  subject  areas  of 
DT&SM  K&I  (e.g.,  for  DTM— research, 
technology  demonstrations;  or  for  DSM — 
systems  engineering,  test  and  evaluation, 
logistics  support), 

•  Use  many  different  storage  media 
(reports,  books,  articles,  electronic  data,  CD- 
ROM  discs,  human  expertise),  and 

•  Are  assembled  to  support  many 
different  DT&SM  purposes  and  functions. 

DT&SM  professionals  need  timely,  selective, 
on-demand  access  to  specific  DT&SM  K&I, 
available  from  the  widely  distributed  K&I 
SOURCEs,  that  are  relevant  to  any  of  die 
five  management  purposes  identified  above 
under  Integrated  DT&SM  Concept. 

Considering  (a)  the  enormous  size  of  the 
body  of  DT&SM  K&I,  and  its  wide  distribu¬ 
tion  throughout  the  DT&SM  community,  in 
thousands  of  SOURCEs  (with  organized 
collections  that  treat  different  K&I  subject 
areas  of  the  body  of  DT&SM  K&I),  and  (b) 
the  pervasive  K&I  access  needs  of  DT&SM 
professionals:  How  best  can  all  of  these 
items  of  DT&SM  K&I  be  identified, 
organized,  classified,  stored,  and  indexed 
at  a  K&I  SOURCE,  for  timely  selective 
on-demand  access  and  retrieval  by 
DT&SM  professionals  throughout  the 
DT&SM  community? 

Our  answer  after  much  research  and  analysis 
is:  Begin  to  implement  a  DT&SM 
Knowledge  Locator  System,  that  has  two 
levels  of  search  for  relevant  K&I: 

•  Macro-level — concerned  with  the 
organizational  name,  location  and  contact 
point  of  each  available  K&I  SOURCE,  the 
main  DT&SM  subject  categories  of  the  K&I 
in  its  K&I  collection,  and  die  media  used  to 
store  the  collection  which  determines  how  the 
collection  will  be  accessed. 

•  Micro-level — concerned  (after  a  per¬ 
tinent  SOURCE  is  located)  with  timely, 
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selective,  on-demand  access  to,  and  retrieval 
of  relevant  K&I  from  the  organized  K&I 
collection  of  the  SOURCE. 

Progress  on  the  first  initiative  proposed 
above — developing  a  DoD  Macro-level 
DT&SM  Thesaurus  Structure  (discussed 
under  Thesaurus  Structure)— is  required  to 
effectively  implement  both  levels  of  the 
DT&SM  Knowledge  Locator  System. 

Progress  on  the  second  proposed  initiative — 
developing  an  on-line  DoD  Macro-level 
DT&SM  Knowledge  Locator  System — is 
contingent  on  using  the  DoD  Macro-level 
DT&SM  Thesaurus  Structure  to  develop  an 
on-line  DoD  Thesaurus  of  Available 
SOURCES  of  DT&SM  K&I.  This 
Thesaurus  of  Available  SOURCES  is  the 
foundation  for  an  on-line  DoD  Macro-level 
Knowledge  Locator  System,  which  any 
DT&SM  professional  can  use  to  search  for 
and  locate  pertinent  SOURCES — those  most 
likely  to  have  a  K&I  collection  which 
contains  K&I  that  are  relevant  to  the  profes¬ 
sional's  task  at  hand.  The  professional  must 
first  find  the  most  likely  useful  SOURCES, 
before  he  or  she  can  employ  IMT  aids  to  get 
timely,  selective,  on-demand  access  to  a 
located  SOURCE'S  organized  collection,  and 
retrieve  specific  relevant  K&I. 

Progress  on  the  third  proposed  initiative — 
that  each  SOURCE  develop  an  on-line 
micro-level  thesaurus  of  the  particular 
DT&SM  K&I  maintained  in  its  own 
organized  K&I  collection — will  take  some 
time  after  the  on-line  DoD  Macro-level 
DT&SM  K&I  Locator  System  is  operational. 
So,  if  an  on-line  micro-level  thesaurus  is  not 
yet  available  in  a  pertinent  SOURCE  found 
with  the  on-line  Locator  System,  the 
DT&SM  professional  can  work  with  the 
SOURCE'S  collection  managers  and  use  the 
SOURCE'S  current  system  for  providing 
users  access  to  its  K&I  collection. 

But  when  the  SOURCE  has  an  on-line 
micro-level  thesaurus  of  its  K&I  collection, 
the  professional  can  use  this  thesaurus  to 
directly  access  and  retrieve  from  the 
SOURCE'S  collection,  specific  documents, 
data,  and/or  consultant  expertise  that  are 
relevant  to  the  professional's  task  at  hand. 


Glossaries  &  Common  Language.  The 
fourth  proposed  initiative — DoD  develop¬ 
ment  and  maintenance  of  an  on-line  DoD 
DT&SM  Glossary — should  begin  in  parallel 
with  the  first  initiative — DoD  developing  a 
DoD  Macro-level  DT&SM  Thesaurus 
Structure — by  adding  each  term  in  the 
macro-level  thesaurus  structure  and  its 
definition  to  the  DT&SM  glossary.  Progress 
should  continue  by  adding  to  the  DT&SM 
glossary,  the  glossary  that  each  SOURCE 
develops  as  a  companion  to  the  SOURCE'S 
thesaurus  of  its  organized  collection  of 
DT&SM  K&I. 

The  DoD  DT&SM  Glossary  and  the  growing 
number  of  SOURCE  glossaries  will  promote 
development  of  an  increasingly  common 
defense  systems  management  language 
throughout  the  DT&SM  community.  The 
increasingly  common  language  will  help 
avoid  serious  errors  and  costly  reworks  due 
to  misunderstandings,  and  will  aid  unambi¬ 
guous  communication  among  DT&SM 
professionals  in  their  coordination  of  tasks 
over  a  system's  life  cycle,  to  get  systems 
management  tasks  and  programs  done  right 
the  first  time. 

Organized  Collections.  The  on-line 
DoD  Thesaurus  of  Available  SOURCES  of 
DT&SM  K&I,  and  thus  the  DoD  Macro¬ 
level  DT&SM  Knowledge  Locator  System, 
will  include  all  available  organized  collections 
of  DT&SM  K&I  that  can  be  found 
throughout  the  DT&SM  community.  Many 
nearly  identical  K&I  collections — developed 
before  the  information  highway  and  current 
IMT  tools  for  remote  access  were  available — 
may  be  unnecessarily  duplicated  in  more  than 
one  SOURCE. 

We  believe  that  USD(A&T)  implementation 
of  our  proposed  initiatives  will  yield 
enormous  savings  as  unnecessary  duplica¬ 
tion  of  major  organized  collections  of 
DT&SM  K&I  are  found  during  development 
of  the  on-line  DoD  Thesaurus  of  Available 
SOURCES  of  DT&SM  K&I.  We  are  sug¬ 
gesting  not  only  enormous  dollar  savings  in 
duplicated  information  dissemination  opera¬ 
tions,  and  in  duplicated  people,  equipment 
and  facilities  throughout  the  DT&SM  com¬ 
munity,  in  DoD  and  other  government 


202 


organizations,  industry  and  academia — but 
also,  opportunity  cost  savings  of  manage¬ 
ment  time  now  spent  searching  for  relevant 
DT&SM  K&I,  of  relevant  K&I  now  missed 
for  lack  of  awareness  of  pertinent  K&I 
SOURCES,  and  from  not  being  able  to  get 
timely  access  to  the  most  relevant  K&I  for 
tasks  at  hand. 

Three  categories  of  organized  DT&SM  K&I 
collections  are  considered  here  for  two 
reasons.  First,  they  provide  examples  of 
what  we  mean  by  organized  K&I  collections. 
Second,  we  believe  that  each  of  these 
categories  are  particularly  useful  in  achieving 
"informational"  integration  of  DT&SM. 
They  should  become  more  widely  known 
through  the  DoD  Macro-level  DT&SM 
Knowledge  Locator  System.  Thus,  their 
K&I  will  be  more  readily  available  to  all 
DT&SM  professionals  in  any  of  the  four 
main  DT&SM  processes  represented  in 
Figure  1  (i.e.,  acquisition,  operational  em¬ 
ployment  support,  disposition,  and  defense 
technology  management),  not  just  to  those 
who  work  regularly  with  each  collection. 

One  category,  distributed  throughout  the 
DT&SM  community,  are  the  K&I  databases 
of  CALS,  initially  (Computer-Aided 
Acquisition  and  Logistics  Support),  but  with 
increasing  emphasis  on  life-cycle  systems 
management,  now  also  (Continuous 
Acquisition  and  Life-cycle  Support).  These 
CALS  K&I  databases  are  the  result  of 
government-industry  development  during  the 
past  five  years,  of  CALS  computer  program 
standards  for  indexing,  storing,  transferring, 
and  retrieving  data  concerning  defense 
systems,  parts,  drawings  and  manufacturing. 
CALS  focuses  mainly  on  DSM  K&I,  but 
these  K&I  are  also  useful  to  DT&SM 
professionals  working  in  DTM. 

Another  category  is  the  Defense  Technical 
Information  Center  (DTIC)  collections, 
including  its  Technical  Reports  collection 
which  also  includes  many  acquisition 
documents,  its  Work  Unit  Information 
System  (WUIS)  and  Independent  Research 
and  Development  (IR&D)  databases,  its 
Information  Analysis  Center  (LAC)  program, 
and  other  Science  and  Technology  (S&T) 
programs.  Formerly,  DTIC's  organized 


collections  focused  mainly  on  traditional 
Science  and  Technology  R&D.  With  the 
October  1994  announcement  of  the  new 
Defense  Science  and  Technology  Strategy 
and  Defense  Technology  Plan,  we  believe 
these  S&T  and  other  collections  will  have  an 
expanded  role  in  DTM,  and  in  increased 
DTM  integration  with  DSM  for  system 
upgrades  throughout  the  life  cycle  of  defense 
systems. 

In  the  third  category,  the  Air  Force 
Acquisition  Model  (AFAM )15  is  a  unique 
powerful  new  software  package,  an  organ¬ 
ized  collection  of  automated  systems 
acquisition  process  how-to  K&I.  AFAM  is  a 
vehicle  by  which  functional  experts  in  the  Air 
Force  Materiel  Command  (AFMC)  provide 
insight  on  procedures,  best  practices,  and 
lessons-leamed  wisdom  needed  to  complete 
over  4000  acquisition  tasks  and  subtasks 
throughout  the  entire  acquisition  process. 
Information  provided  for  each  task  includes: 
acquisition  relationships  (past,  current  and 
future  tasks),  references  to  additional  K&I, 
wisdom  (best  practices,  lessons-leamed, 
expert  opinion),  tools  (templates,  courses, 
software),  and  a  timeline  estimate  of  task 
length. 

The  AFAM  software  package  also  includes  a 
supplemental  package  (another  organized 
collection),  called  the  Air  Force  Acquisition 
Model  Supplement  (AFAMSUP).  It  is  a  text 
retrieval  system  that  currently  houses  over 
113  current  documents — DODD  5000.1  and 
DODI  5000.2,  DoD  MIL-STDs,  Air  Force 
Regulations  and  AFMC  acquisition 
documents. 

AFAM  is  designed  to  bring  far  more 
comprehensive  "expert"  experience  into  Air 
Force  acquisitions  when  it  is  most  needed — 
at  the  beginning.  AFAM  runs  on  an  IBM- 
compatible  personal  computer,  as  a  stand 
alone  or  on  a  Local  Area  Network  (LAN). 
The  numerical  statistics  for  tasks  and  docu¬ 
ments  stated  above  are  for  AFAM  Version 
2.0,  1  September  1994.  Each  subsequent 
version,  released  every  180  days,  will  add 
more  enhancements  and  data  to  reflect  the 
best  current  practices  and  corporate 
knowledge  changes. 
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We  believe  that  AFAM  is  a  model  that  other 
Military  Services  and  agencies  could  adapt  in 
order  to  identify  their  respective  tasks  and 
DT&SM  documents  useful  to  their  program 
management  offices  and  others  in  the  life- 
cycle  management  of  their  defense  systems. 

We  also  see  the  AFAM  model  of  acquisition 
tasks  and  its  DoD  5000.1  and  5000.2 
documents,  as  very  useful  in  our  first 
proposed  initiative — developing  a  DoD 
Macro-level  DT&SM  Thesaurus  Structure. 
The  DoD  documents  would  be  sources  of 
terms  and  concept  definitions,  and  the  task 

statements  would  help  identify  associative 
relationships  among  processes  that  are 
otherwise  unrelated,  but  are  required  together 
to  complete  a  task. 

In  addition,  the  AFAM's  easy  customer  feed¬ 
back  to  shape,  refine  and  enhance  future  re¬ 
leases  of  AFAM  is  a  good  model  for  getting 
feedback  to  continually  improve  the  IMT- 
aided  management  tools.  For  example:  to 
increase  the  usefulness  of  the  terms  used  in 
successive  releases  of  the  DoD  Macro-level 
DT&SM  Thesaurus  Structure,  and  to  find 
additional  available  K&I  SOURCES  to 
expand  and  improve  the  DoD  Thesaurus  of 
Available  SOURCES  of  K&I 

SUMMARY 

The  present  world  and  national  environment 
requires  substantial  changes  in  the  process  of 
managing  defense  systems — changes  from 
managing  separately  the  four  processes  por¬ 
trayed  on  Figure  1,  (DTM  and  the  three 
successive  DSM  stages  of  each  system's  life 
cycle),  to  integrated  Defense  Technology  and 
Systems  Management  (DT&SM)  throughout 
every  system's  life  cycle. 

Recent  advances  in  computer  and  other 
information  management  technologies  (IMT) 
can  make  "informational"  integration  of 
DT&SM  a  practical  reality.  Truly  integrated 
DT&SM  will  not  only  reduce  costs,  but  also 
best  assure:  increased  rationalization,  stan¬ 
dardization  and  interoperability  of  DT&SM 
processes  and  products,  the  continued  tech¬ 
nological  superiority  of  our  defense  systems, 
and  sustained  system  operational  suitability 
and  effectiveness  in  all  stages  of  their  life 


cycles.  It  would  also  help  AFMC's  IWSM 
develop  a  common  language  for  life-cycle 
management  of  Air  Force  systems. 

To  start  progress  toward  this  goal,  we  pro¬ 
pose  four  initiatives  by  the  Under  Secretary 
of  Defense  for  Acquisition  and  Technology 
(USD(A&T))  to  begin  development  of  LMT- 
aided  tools  that  will  enable  achievement  of 
integrated  DT&SM.  Ultimately,  these  tools 
will  enable  any  DT&SM  professional  to 
directly  access  pertinent  K&I  SOURCES 
throughout  the  DT&SM  community,  and 
selectively  retrieve  specific  DSM  or  DTM 
K&I  that  are  relevant  to  the  professional's 
task  at  hand.  The  initiatives  are: 

1.  Initiate  immediate  development  of  an 
on-line 5  DoD  Macro-level  DT&SM  Thesau¬ 
rus  Structure.  This  thesaurus  structure  is  the 
key  management  tool  for  integrating 
DT&SM,  beginning  with  its  use  and  applica¬ 
tion  in  the  following  proposed  initiatives: 

2.  Then,  initiate  development  of  an  on¬ 
line  DoD  Macro-level  DT&SM  Knowledge 
Locator  System,  based  on  a  DoD  Thesaurus 
of  Available  SOURCES  of  DT&SM  K&I 
that  are  distributed  throughout  the  DT&SM 
community,  in  the  DoD  and  other  govern¬ 
ment  organizations,  industry  and  academia. 
Once  operational,  any  DT&SM  professional 
can  use  the  Locator  System  to  quickly  locate 
increasing  numbers  of  available  SOURCES 
of  DT&SM  K&I  that  may  have  specific  K&I 
the  professional  needs  for  a  task  at  hand. 

3.  Encourage  evolutionary  extension  of 
the  on-line  DoD  Macro-level  DT&SM 
Thesaurus  Structure  down  into  every  K&I 
SOURCE  included  in  the  Knowledge 
Locator  System,  by  encouraging  every 
SOURCE  to  develop  an  on-line  micro-level 
thesaurus  and  a  companion  glossary  of  the 
special  K&I  available  in  the  SOURCE. 
These  new  on-line  micro-level  thesauri  will 
enable  any  DT&SM  professional  to  use  the 
information  highway  (for  example,  the 
DT&SMlink  suggested  above),  not  only  to 
locate  pertinent  SOURCES  of  likely  useful 
K&I,  but  also  to  directly  access  the 
SOURCE'S  on-line  thesaurus  and  retrieve 
K&I  that  are  relevant  to  a  task  at  hand. 

4.  Initiate  DoD  development  and 
maintenance  of  an  on-line  DoD  DT&SM 
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Glossary  (and  subset  area  glossaries)  that 
provides  current  authenticated  definitions  for 
every  term  in  the  DT&SM  Thesaurus 
Structure.  Each  companion  glossary  (for  the 
micro-level  thesaurus  that  each  SOURCE 
develops  for  its  collection  of  K&I)  should 
also  be  added  to  the  on-line  DoD  DT&SM 
Glossary.  As  these  glossaries  become  more 
comprehensive  they  will  support  develop¬ 
ment  of  an  increasingly  common  defense 
systems  management  language  throughout 
DT&SM  community.  This  wiU  enable  better 
communication  with  understanding  among 
DT&SM  professionals  and  more  effective 
coordination  to  get  systems  management 
tasks  and  programs  done  right  the  first  time. 

Thus,  the  DoD  DT&SM  macro-level 
thesaurus  structure  will:  support  development 
of  DT&SM  on-  and  off-ramps  to  the  infor¬ 
mation  highway  (e.g.,  Knowledge  Locator 
System),  aid  development  of  guidebooks  for 
DT&SM  K&I  SOURCES  (e.g.,  micro-level 
K&I  thesauri),  promote  development  of  a 
common  DT&SM  language  (e.g.,  DT&SM 
glossaries),  and  enable  more  effective  use  of 
information  management  technologies  to 
achieve  integrated  life-cycle  management  of 
defense  technology  and  defense  systems. 
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ABSTRACT 

In  1988,  the  DoD  implemented  an  initiative 
called  Concurrent  Engineering  (CE)  to 
improve  weapon  system  quality  while  at  the 
same  time  reducing  both  cost  and  schedule 
risk.  The  United  States  Air  Force  (USAF) 
expanded  CE  concepts  to  include  all 
disciplines,  not  just  technical,  and  developed 
the  concept  known  as  Integrated  Product 
Development  (IPD).  The  Air  Force  Systems 
Command  successfully  applied  IPD  to  one 
of  its  most  important  weapon  system 
programs:  now  the  F-22  Advanced  Tactical 
Fighter  (ATF).  Over  the  past  four  years, 
however,  the  F-22  Team1  recognized  that  a 
plan  for  continually  improving  IPD 
implementation  is  necessary  to  ensure  a 
quality  weapon  system  design.  This  paper 
describes  the  F-22  IPD  Implementation 
Improvement  Plan.  It  outlines  the  tool  and 
process  used  by  the  F-22  Team  to  measure 
and  improve  its  implementation  of  IPD  and 
the  overall  quality  of  the  F-22  weapon 
system. 

EVOLUTION  OF  IPD 

Concurrent  Engineering.  In  1986,  the 
President’s  Blue  Ribbon  Commission  on 
Defense  Management  noted  that  weapon 
systems  take  too  long  to  develop,  cost  too 
much  to  produce,  and  often  do  not  perform 
as  promised  or  expected.2  At  the  same  time, 
private  industry  was  attributing  its  success 
in  the  competitive  global  market  to  its  use  of 
new  integrated  design  initiatives  such  as 
concurrent  engineering  (CE).3  US  industry 


reported  improved  quality  of  their  products 
and  services,  decreased  costs,  and  improved 
adherence  to  shorter  schedules  by  the  use  of 
concurrent  engineering  approaches.  In 
1986,  the  Under  Secretary  of  Defense  for 
Acquisition  USD(A)  chartered  the  Institute 
for  Defense  Analysis  (IDA)  to  examine  the 
concept  of  CE.4  CE  has  its  roots  in  the 
integrated  design  and  production  practices 
which  emerged  in  the  Japanese  auto  industry 
following  World  War  II.5  CE  is  defined  as: 

A  systematic  approach  to  the  integrated, 
concurrent  design  of  products  and  their 
related  processes,  including 
manufacturing  and  support.  This 
approach  is  intended  to  cause 
developers,  from  the  outset,  to  consider 
all  elements  of  the  product  life  cycle 
from  conception  through  disposal, 
including  quality,  cost,  schedule,  and 
user  requirements .6 

The  IDA  studied  CE  as  implemented  by 
fourteen  major  US.  corporations  and 
conducted  workshops  to  learn  about  the 
various  approaches  taken  to  apply  CE  in 
industry.  In  the  final  report,  the  major 
recommendation  by  the  IDA  was  that  the 
DoD  take  positive  steps  to  encourage  use  of 
CE  in  weapon  systems  acquisition.7  The 
IDA  concluded  that  companies  which 
implemented  CE  practices  produced  higher 
quality  products  at  lower  cost  and  in  less 
time  than  before.  Following  this 
recommendation,  the  DoD  and  Service 
acquisition  secretaries  required 
implementation  of  CE  into  the  DoD's 
weapon  systems  acquisition  process.8’9 
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IPD  in  the  USAF.  The  Air  Force  Systems 
Command10  expanded  on  the  CE  concepts 
to  include  all  disciplines  (e.g.  finance, 
logistics,  etc.),  not  just  technical,  and  named 
their  implementation  concept  of  CE 
“ Integrated  Product  Development ’  or  IPD. 
The  USAF  defines  IPD  as: 

A  philosophy  that  systematically  employs 
a  teaming  of  functional  disciplines  to 
integrate  and  concurrently  apply  all 
necessary  processes  to  produce  an 
effective  and  efficient  product  that  satisfies 
customers '  needs. 1 1 

This  modified  concept  of  CE  was 
implemented  by  Air  Force  Materiel 
Command  (AFMC).  Certain  weapon 
development  programs  including  the  F-22 
fighter  weapon  system  (others  were  F-15,  F- 
16,  and  B-2)  were  identified  within 
Aeronautical  Systems  Division  (ASD)  as 
test  beds  for  IPD.  This  expanded  concept  of 
CE  has  been  used  on  the  F-22  Advanced 
Tactical  Fighter  (ATF)  program  since  the 
award  of  the  Engineering  &  Manufacturing 
Development  Phase  contract  in  1991. 

Tenets  of  IPD.  The  IPD  philosophy 
embodies  eight  key  tenets.12  The  F-22 
Team  used  these  tenets  to  guide  its 
implementation  of  IPD. 

(1)  Cultural  Change:  Embracing  the  IPD 
philosophy  requires  purposeful,  multi- 
disciplined  teamwork.  The  priority  of  focus 
for  IPD  should  be: 

a)  The  customer 

b)  The  product 

c)  The  process 

d)  Constraints,  ...and  then 

e)  Organizational  structure 


The  first  step  requires  focus  on  the 
customers'  needs  to  determine  what  the 
product  will  be.  The  necessary  processes 
are  then  implemented  in  order  to  deliver  the 
product  to  the  customer.  The  customer,  the 
nature  of  the  product,  and  the  organization 
structure  provide  the  constraints  applied  to 
the  multi-disciplined  team.  Constraints  may 
require  modification  to  existing  processes, 
development  of  new  processes,  and  changes 
to  organizational  structure. 

(2)  Product  Focus:  IPD  requires  a  product 
focus  and  a  complete  understanding  of  the 
processes  required  to  optimize  the  product. 

(3)  Up-Front  Planning:  The  life  cycle  of  a 
product  or  process  will  be  integrated 
through  comprehensive,  up-front  planning 
that  must  include  all  functions,  customers, 
and  suppliers. 

(4)  Right  People.  Right  Place.  Right  Time: 
All  functions  that  impact  the  achievement  of 
the  customer's  requirements  should  be 
applied  concurrently,  in  a  team  fashion, 
throughout  the  life  of  a  product  or  process. 

(5)  Teamwork  and  Communication:  People 
must  function  as  a  team.  Team  success, 
facilitated  by  rapid,  open  communication, 
must  be  emphasized  and  rewarded. 
Management  relationships  must  be 
developed  which  are  consistent  with  and 
focused  on  achieving  the  team's  measurable 
goals  and  objectives.  Integrated  Product 
Teams  (IPTs)  are  the  heart  of  IPD.  The 
F-22  IPT  structure  is  shown  in  Figure  1.0. 
Key  characteristics  of  IPTs  are:13 

a)  Team  is  set  up  to  produce  a  specific 
product  or  service. 
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b)  Multidisciplinary  -  all  team 
members/functions  working  together 
towards  a  common  goal. 

c)  Members  have  mutual,  as  well  as 
individual  accountability. 

d)  Integrated,  concurrent  decision  making. 

e)  Empowered,  within  specific  product  or 
service  goal,  to  make  decisions. 

f)  Planned  integration  among  teams  towards 
system  goal. 

(6)  Empowerment:  Decisions  must  be 
driven  to  the  lowest  possible  level 
commensurate  with  risk.  Resources  should 
be  allocated  at  levels  consistent  with 
authority,  responsibility,  and  ability  of  the 
people. 

(7)  Seamless  Management  Tools:  A 

framework  must  be  established  which 
relates  products  and  processes  at  all  levels  to 
demonstrate  dependency  and 

interrelationships.  This  hierarchical 
interrelationship  (e.g.,  Systems  Engineering 
Master  Plan  methodologies)  must  be 
understood  and  appropriate  partnerships 
established  to  ensure  that  all  decisions  are 
optimized  toward  the  ultimate  user's  end 
product. 


(8)  Integration  Throughout  the  Life  Cycle: 
IPD  will  encompass  all  products  and 
processes,  regardless  of  the  point  in  their  life 
cycle. 

7  CHALLENGES  to  IPD 
IMPLEMENTATION 

Implementation  of  IPD  poses  several  unique 
challenges  to  success.  Government  and 
industry  leaders  have  outlined  seven  barriers 
to  implementing  CE  concepts  (i.e.  IPD) 
within  the  DoD.14  These  are: 

1)  Trust  5)  Contracting 

2)  Leadership  6)  Measurement 

3)  Resistance  7)  Understanding 

4)  Resources 

The  F-22  Team  has  experienced  challenges 
1-5  and  met  them  with  initial  success. 
Measurement  of  IPD  implementation  and 
fostering  widespread  understanding  of  IPD 
are  ongoing  efforts. 

How  the  F-22  Team  Met  Challenges  1-5. 
During  the  initial  implementation  of  IPD 
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within  the  F-22  Team,  challenges  1-5  had  to 
first  be  overcome.  Before  any  progress 
towards  IPD  implementation  could  even 
take  place,  trust  had  to  be  established 
between  all  F-22  Team  members.  Trust  was 
established  by  a  long  term  commitment  to 
making  and  keeping  agreements, 
maintaining  credibility,  and  maintaining 
openness.  Challenge  2,  leadership,  was 
required  at  all  levels  of  acquisition  to 
support  IPD.  To  provide  this,  leaders  were 
empowered  at  the  lowest  levels  possible  to 
make  the  right  decisions.  This  strengthened 
team  leadership  at  the  worker/action  level. 
Using  this  method,  information  is  shared 
and  issues  are  resolved  at  the  lowest 
practical  Integrated  Product  Team  (IPT) 
level.  Challenge  3,  resistance  by  functional 
groups  (i.e.,  engineering,  finance  and 
contracting)  was  eased  by  rigorously 
applying  tenet  five,  teamwork  and 
communication  through  IPTs.  All  functions 
that  impacted  the  achievement  of  the 
customers  requirements  were  applied 
concurrently  in  IPTs.  Team  members  take 
ownership  in  each  action  and  outcome  of  the 
IPT.  Early  resource  commitment,  (i.e., 
time,  people,  and  funding)  at  adequate  levels 
(Challenge  4)  was  performed  by  identifying 
the  requirement  for  IPD  at  the  beginning  of 
the  engineering  and  manufacturing 
development  phase  (EMD).  Resources  for 
the  IPTs  were  committed  at  this  time.  The 
F-22  Team  overcame  challenge  5, 
contracting,  by  changing  the  contract  to 
support  IPD  and  CE  concepts.  In  the  initial 
request  for  proposal  the  requirement  to 
implement  IPD  was  first  listed  in  the 
Statement  of  Work  (SOW).  It  stated 
“...using  the  Integrated  Product 
Development/ Concurrent  Engineering 
approach,  the  contractor  shall  accomplish 
the  tasks  required  to  perform  systems 
engineering  and  program  management,  and 
maintain  technical  control  of  the  F-22 


Weapon  System.  ”  This  established  the  use 
of  IPD  as  a  requirement  by  each  contractor. 

The  Gap  -  Challenges  6-7.  The  final 
challenges  to  implementing  IPD  involved 
fostering  universal  understanding  of  the 
philosophy,  processes,  tools  and  applications 
of  IPD  and  measuring  progress  of  IPD 
implementation.  The  AFMC  IPD  guide 
provides  an  evaluation  scale  to  aide 
measurement  and  understanding  of  IPD 
concepts.15  The  F-22  Team  used  this  scale 
as  a  starting  point  to  develop  its  own 
understanding  and  measurement  of  its  IPD 
implementation  progress.  After  an  initial 
trial  with  the  IPD  guide  evaluation  scale,  the 
F-22  Team  found  a  need  to  develop  their 
own  scale  or  “tool”  which  was  more  product 
oriented  and  gave  a  clearer  understanding  of 
what  IPD  should  look  like  in  the 
organization.  The  F-22  Team  needed  to 
develop  a  process  and  tool  to  measure  and 
improve  the  implementation  of  IPD  and  the 
overall  quality  of  the  F-22  weapon  system 
design.  These  final  challenges  were  recently 
taken  on  by  the  F-22  Team  in  the  form  of 
the  F-22  IPD  Implementation  Improvement 
Plan. 

THE  F-22  IPD  IMPLEMENTATION 
IMPROVEMENT  PLAN 

A  Structured  Approach  to  Measurement 
&  Understanding.  With  the  challenges  of 
measurement  and  understanding  in  mind, 
the  SPO  Director  chartered  a  project  team  to 
develop  an  assessment  tool  and  a  structured 
process  to  measure  and  continuously 
improve  the  F-22  Team's  implementation  of 
IPD.  Eight  project  team  members,  chosen 
to  represent  the  SPO  and  each  of  the  four 
primary  contractors,  developed  an  IPD 
Implementation  Template  (Figure  2.0)  as 
the  assessment  tool  and  a  process  for 
continuous  improvement  (Figure  3.0).  Key 
aspects  of  this  process  are  the  use  of  a 
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template  to  assess  progress,  development  of 
action  plans  for  improvement,  and  briefings 
at  Program  Management  Reviews  (PMRs)16 
to  status  progress  and  facilitate  exchange  of 
best  practices  across  the  F-22  Team. 

Development  of  the  IPD  Implementation 
Template.  The  structure  of  the  template 
draws  upon  the  work  done  by  the 
CALS/Concurrent  Engineering  Working 


template  (Figures  2. 1-2.4)  breaks  down  each 
of  the  top-level  categories  into 
subcategories,  providing  the  detail  necessary 
to  perform  an  assessment.  The  project  team 
developed  wording  for  the  template  by 
gathering  inputs  and  getting  feedback  from 
the  IPTs.  Project  team  members  solicited 
comments  from  IPT  leaders  through 
personal  interviews  and  a  series  of  trial  runs 
briefed  by  the  IPTs  at  the  August  and 


STAGE  I 

mam 

1.0 

Organization 

Structure 

-  no 

interdisciplinary 

teams 

-  little 

customer/supplier 

involvement 

-  decision  making 
distributed  through 
functional 
organization 

-  ad  hoc  process 
action  teams  (PATs) 

&  tiger  teams 

-  decision  making 
resides  with  person 
who  chartered  the 
team 

-  integrated 
engineering  design 
teams 

-  team  has  authority 
to  make  engineering 
decisions 

-  multifunction 
teams 

-  customer/supplier 
is  full  team  member 

-  decision  making 
authority  inc] udes 
all  needed 
disciplines 

-  team  has  separate 
budget,  resource 
authority  A 
responsibility 

2.0  System 
Integration 

•  integration  A 
planning  handled 
primarily  by  upper 
management 

-  integration 
performed  serially 
by  function 

-  system  engineering 
process  applied  to 
improve  integration 

>  integration  is  a 
principle  element  of 
each  team 's  mission 

3.0  Processes 

-  processes 
institutionalized  in 
functional 
organization 
structure 

-  processes 
becoming 
team/project 
oriented 

-  best  practices 
applied  to  improve 
critical  processes 

-  all  processes 
identified, 
documented  and 
continuously 
Improved 

4.0  Human 

Resource 

Management 

•  HR  management 

functionally 

oriented 

-  project  teams 
formed,  but  HR 
management  still 
functionally 
oriented 

-  HR  management 
processes  recognize 
team  needs 

-  HR  management 
processes  fully 
adapted  to  team 
needs 

®- CURRENT  STATUS 

g[-  DESIRED  STATUS 

Figure  2.0  Top  Level  Template  -  IPD  Implementation  Improvement  Plan 

Group  -  Electronic  Systems  Task  Group.17 
The  template  is  a  matrix,  with  the  rows 
representing  categories  and  the  columns 
representing  stages  of  implementation.  The 
project  team  developed  the  template  in  two 
levels  for  ease  of  use:  a  top  level  template, 
and  a  reference  level  template.  The  top 
level  template  (shown  in  Figure  2.0)  is  used 
as  a  "quick-look"  assessment  chart  suitable 
for  briefing  purposes.  The  reference  level 


October  PMRs.  In  their  briefings,  IPTs 
were  required  to  brief  their  assessment,  their 
action  plans  for  improvement,  and  best 
practices  that  could  be  shared  with  other 
IPTs. 

Template  Description.  The  stages  of  the 
template  (the  columns)  represent  the 
progressive  degrees  of  implementation  of 
IPD.  The  wording  in  each  stage  describes 
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what  an  organization  might  look  like  at  that 
stage.  In  general,  Stage  I  represents  an 
organization  that  has  not  yet  implemented 
IPD,  while  Stage  IV  represents  an 
organization  that  has  implemented  IPD  to 
the  fullest  extent  conceived  at  this  time. 
The  intermediate  stages  are  part  of  a 
continuum  between  the  two  extremes.  Stage 
N,  or  Stage  Next,  represents  a  stage  of 
implementation  yet  to  be  defined.  As  the 
IPTs  reach  Stage  IV,  they  obtain  a  better 
understanding  of  IPD  concepts  and  will 
begin  to  look  ahead  to  further  improve  their 
IPT.  As  IPTs  increase  IPD  understanding 
and  strive  for  continuous  improvement,  they 
will  begin  filling  in  their  own  words  for 
Stage  N.  Once  several  of  the  IPTs  have 
defined  their  own  Stage  N,  the  template 
project  team  will  gather  the  inputs  and 
publish  standard  wording  for  Stage  N.  This 
new  stage  will  become  Stage  V,  and  another 
blank  Stage  N  will  be  added  to  the  right  side 
of  the  chart.  In  this  way,  organizations  will 
always  be  stretching  to  find  new  ways  to 
continuously  improve.  The  empty  blocks  in 
the  reference  level  template  are  areas  where 
the  IPTs  and  project  team  could  not  describe 
intermediate  levels  of  the  IPD 
implementation  continuum.  These  blocks 
may  be  filled  in  later  as  IPTs  use  the 
templates  to  develop  a  better  understanding 
of  IPD. 

The  categories  of  the  template  (the  rows) 
represent  the  aspects  of  IPD  the  team 
selected  to  measure  and  are  aligned  with  the 
eight  IPD  tenets  identified  earlier.  The 
following  is  a  description  of  each  category 
of  the  template. 

Category  1.0  -  Organization  Structure:  IPD 
tenet  4  states  that  “All  functions  that  impact 
the  achievement  of  the  customer’s 
requirements  should  be  applied 
concurrently,  in  a  team  fashion,  throughout 


the  life  of  a  product  or  process.” 
Organization  structure  is  often  the  first,  and 
most  recognizable,  change  made  to 
implement  IPD.  IPD  represents  a  shift  in 
the  project  management  approach  from 
functional  orientation  to  product  orientation. 
Therefore,  creating  the  proper  team 
environment  for  IPD  implementation  is 
imperative.  The  very  essence  of  IPD  is  that 
all  necessary  disciplines  are  represented  on 
the  team,  so  that  their  inputs  can  be  made  to 
the  product.  In  general,  IPTs  are  the 
predominant  organization  structure  used  to 
implement  IPD.  The  IPT  is  simply  a  team 
of  the  right  people.  This  category  examines 
the  composition  of  the  team,  involvement  of 
customers  and  suppliers,  and  the  interaction 
of  the  team  members.  Empowerment, 
another  IPD  tenet,  is  also  measured  by 
assessing  the  team’s  scope  of  accountability, 
authority,  and  responsibility. 

Category  2.0  -  System  Integration:  Based 
on  F-22  experience,  one  of  the  biggest 
challenges  of  IPD  is  integration  across  IPTs. 
While  IPTs  are  focused  on  delivering  their 
product,  careful  attention  must  be  paid  to 
integrating  across  teams  so  that  decisions  are 
made  to  optimize  the  final  product  for  the 
end-user.  Integration  and  planning  methods 
must  be  developed  and  applied  across  the 
teams  to  achieve  final  product  optimization  . 
In  addition,  the  program  office  must  have  a 
clear  set  of  customer  requirements,  as  well 
as  a  process  for  capturing  evolving 
requirements,  to  understand  what  the  end- 
user  needs.  By  examining  the  planning 
methods  and  processes  used  to  accomplish 
effective  requirements  definition  and 
integration,  this  category  assesses  the  tenets 
of  Up-Front  Planning,  Integration 
Throughout  the  Life-Cycle,  and  Seamless 
Management  Tools. 
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Category 

STAGE  I 

STAGE  II 

STAGE  III 

STAGE  IV 

STAGE  N 

1.1  Team 

Composition  -  who  is 

-  task  distributed  to 
functional  organizations 

-problems/special 
initiatives  lead  to 
creation  of  ad  hoc 
teams  staffed  by 
functional  experts 

-  dedicated  design 
team  composed  for 
duration  of  project 

-  dedicated 
comprehensive 
team  composed  for 
life-cycle  (engr, 
mfg,  QA,  logistics, 
business  mgt,  etc.) 

-  skill  mix  evolves 
with  program  phase 

on  the  team 

1.2  Team  Member 
Interaction  -  how 

-  individuals  focused  on 
specific  task  or  discipline 
within  their  functional 
organization 

-  tendency  to  sub 
optimize  to  strongest 
discipline 

-  multi  discipline 
perspective 
(aerodynamics, 
structure,  avionics, 
controls,  etc.) 

-  multi-functional 
perspective  (Eng., 
logistics,  mfg., 
business  mgt,  etc.) 

team  member's  work 
together 

13  Decision  making- 
scope  of 
accountability, 
authority  and 
responsibility 

-  management  plays 
directive  role 

-  short  term 

focus/payback  dominates 

-  management 
delegates  decision 
making  to  ad  hoc 
team  for  specific 
problem 

resolution/initiative 

-  team  has 
authority  to  make 
engineering 
decisions 

-  management 
clearly  defines  team 
decision 
boundaries, 
authority,  & 
accountability 

-  team  has  separate 
budget,  resource 
authority  & 
responsibility 

-  c/s  involvement  limited 
to  formal  reviews 

-  team  actively 
solicits  c/s  inputs 

-  c/s  is  active  team 
member, 

participates  in  team 
decisions 

Involvement  - 

customer  includes 
internal,  external,  and 
end  users  (ACC, 

ALC,  operational 
units  etc.)  Suppliers 
include  internal  and 
external. 

Figure  2.1  Reference  Level  Template  -  Category  1.0  " Organization  Structure" 

Category  3.0  -  Processes:  IPD  tenet  2  states, 
“IPD  requires  a  product  focus  and  a 
complete  understanding  of  the  processes 
required  to  optimize  the  product.”  While 
IPTs  focus  on  the  product,  process 
management  is  essential  to  the  effective 
operation  of  the  IPTs.  The  functional 
organizations  play  the  key  role  as  “process 
owners”  in  establishing,  maintaining,  and 
improving  processes  used  by  the  product 
teams.  Effective  process  management  helps 


mitigate  the  impact  of  personnel  turnover 
and  aids  in  consistent  integration  across  the 
IPTs.  As  identified  by  IPD  Tenet  5, 
communication  is  essential  to  the  success  of 
IPD.  Information  must  be  exchanged 
horizontally  and  vertically  throughout  the 
organization  to  ensure  effective  integration 
across  the  IPTs.  This  category  examines 
key  organizational  processes  and  the 
methods  used  to  improve  these  processes. 
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Category  4.0  -  Human  Resource 
Management:  As  indicated  by  IPD  tenet  5, 
for  IPD  to  succeed  “Management 


relationships  must  be  developed  which  are 
consistent  with  and  focused  on  achieving  the 
team’s  measurable  goals  and  objectives." 


Category 

STAGE  I 

STAGE  II 

STAGE  III 

STAGE  TV 

STAGE  N 

2.1  Integration  - 
How  integration 
occurs. 

-  product  integration  by  a 
few  project  engineers 

-  small  team  of 
systems  engineers 
facilitate  integration 
process 

-  team  responsible  for 
integration 

-  integration 
processes  defined 
and  used 

-  teams  accountable 
for  interface  mgt 

2.2  Plan/ 
Methodoloev  - 
process  used  to 
plan  &  monitor 
the  program 
(short  term) 

-  planning  expressed  as 
individual  detailed  tasks 
and  the  project  plan  is  the 
collection  of  these 
detailed  tasks 

-  an  aligned  view  of 
the  project  achieved 
through  work 
breakdown  structure 
methodologies 

-  tasks  are  concurrent 
&  interdependent 

-  interrelationship 
between  tasks  must 

be  known  and 
planned  accordingly 

-  task  identification 
iteratively  improved 
throughout 
development 

2.3  Program 
Master  Plan 
(PMP)  - 
organization's 
planning  (long 
term)  to  address 
future  program 
path  and 
potential  issues 
(e.g.,  technology 
transition,  P3I) 

-  no  documented  long 
term  plans 

-  PMP  exists  but  sits 
on  the  shelf 

-  long  term  planning 
documented  by 
management  but  not 
flowed  down 

-  PMP  is 
working/living 
document,  strategic 
outlook 

-  guides  teams 
product/tactical 
decisions  to  lowest 

levels 

-  3  to  10  plus  year 
outlook  as 
appropriate  for 
product 

2.4  Reauirements 

-  customer  detines 
requirements  to  every 
supplier  (e.g.,  ACC  to 
SFO,  SPO  to  Ktr,  Ktr  to 
vendor) 

-  design  flexibility  is 
limited  by  detailed 
requirements  definition 

-  requirements  may  not 
be  fully  understood  by 
designers 

-  requirements 
introduced  before 
adequate  maturity 
resulting  in  excessive 
scrap  &  rework 

-  requirements 
traceability  tools  not  used 

i 

-  requirements 
understood  and 
accepted  by  designers 

-  derived  requirements 
not  traceable  to  higher 
specification 

-  requirements 
challenges  are 
encouraged  by  the 
customer  as  the 
design  matures 

-  aware  of  new  / 
changing  requirement 
however,  reactive 
response  incurs  cost 
&  schedule  impacts 

-  requirements  linked 
across  specifications 

-  joint,  iterative, 
requirements 
definition  through 
all  customer/ 
supplier  (c/s) 
relationships  (e.g., 
ACC,  SPO,  Ktr, 
vendor) 

-  anticipating 
change  and 
responding 
effectively  to  new 
requirements  with 
minimum  cost  & 
schedule  impact  due 
to  strategic  outlook 

-  adequate  time 
provided  up  front 
before  design 
release  to  mature  all 
requirements 

-  upward/downward 
traceability  database 
supports  validation/ 
verification 

Definition  - 
process  for 
capturing 
customer 
requirements  and 
translating  them 
into  the 

product/process 
specification  to 
satisfy  complete 
life  cycle  needs 

Figure  2.2  Reference  Level  Template  -  Category  2.0  "System  Integration" 
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3.1  Communication 
-  information 
exchanged  between 
and  within 
organization 
components 


3.2  Reviews  - 


process  for 
assessing  product 
process 


33  Information 


Resource  M 


process  for  timely 
exchange  of 
accurate 

information  which 
efficiently  utilizes 
communication/ 
computing 
technology 


3.4  Tool 
Development  & 


Usage  - 

management  tools 
(e.g.,  cost, 
schedule, 
performance) 

-  technical  tools 
(e.g.,  design  & 
analysis) 


STAGE  I 


-  up  and  down  functional 
chain 

-  information  received  is 
functional  specific,  lack 
information  outside  the 
functional  discipline 


STAGE  II  |  STAGE  III 


-  schedule- driven  process 
and  product  critiques 

-  each  discipline 
schedules  series  of 
reviews  with  little 
consideration  of  cross¬ 
discipline  trade-offs 


-  local  individual  data 
management 

-  data  requested  as 
needed  from  other 
disciplines 

-  few  data  management 
tools 


-  Minimum  usage  of 
standard  tools 


-  functional  lines  of 
communication 
blurred,  no 

communication  norms 
established 

-  ad  hoc  team  gathers 
information  to 
complete  their  project, 
communication 
outside  the  ad  hoc 
team  suffers 


-  event  driven  reviews 
ensure  complete 
design  package,  rather 
than  pieces,  is 
reviewed 

-  reviews  are  multi- 
disciplined  with  total 
team  participation 


-  data  is  managed  by 
the  multi-discipline 
team 

-  data  is  structured  for 
sharing  within  the 
team 

-  tools  are  in  place  for 
team  members  to 
retrieve  the  data  they 
need  to  make 
optimum  tradeoffs 


-  Some  usage  of 
automated  tools 


-  communication 
within  teams  but 
limited  between 


communication 
norms  established 
-  some  incomplete 
and  untimely 
information,  junk 
mail  impedes 
processing  of 
relevant  information 


-  immediate  issue 
resolution 

-  process  in  place  to 
gather  real  time 
input  to  make 
decisions  versus 
waiting  for  a  formal 


-  data  is 
automatically 
provided  to  users 
that  need  the 
information 

-  smart  tool  in  place 
for  data  generator  to 
automatically 
communicate 
information  to 
impacted  team 
members 


-  Automated  tools 
developed  for 
critical  processes 


STAGE  rv 


-  information  is 
exchanged  horizontally 
&  vertically  throughout 
the  organizations 

-  everyone  receives  all 
the  information  they 
require  when  it  is 
needed,  no  junk  mail 


-  non-interruptive 
internal  status  reviews 
(ISRs)  used  to  ensure 
correct  processes  are 
observed 

-  ISRs  develop 
strategies  for  product  & 
process  improvement 
rather  than  focusing  on 
the  details  of  the  design 


-  repository  of  working 
data  available  across 
entire  team  enterprise 

-  appropriate  data 
management  system  in 
place  with  efficient  and 
accurate  interpretation 
of  queries 


-  Initiate  integration  of 
toots  across  teams 


STAGE  N 


Figure  23  Reference  Level  Template  -  Category  3.0  "  Processes " 


There  is  a  clear  recognition  that  Human 
Resource  Management  must  evolve  to 
facilitate  IPD.  However,  Human  Resource 
Management  is  typically  the  last  function  to 
change  practices  to  meet  the  needs  of  IPD 
due  to  the  difficulty  of  changing  personnel 
systems.  In  creating  the  subcategories,  the 
project  team  chose  aspects  of  human 
resource  management  that  must  change  to 
facilitate  operations  in  the  IPD  environment. 


about 


Template.  There  are  several  key  concepts 
to  understand  about  the  development  of  the 
Template  and  its  use. 

1)  The  template  was  purposely  developed 
to  be  subjective,  rather  than  objective.  The 
project  team  felt  there  would  be  more 
openness  and  honesty  in  the  assessments  if 
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scores  were  not  attached  to  the  template. 
This  reinforces  the  SPO/Contractor  team 
relationship  as  well.  Despite  the  F-22 
Team’s  history  of  close  SPO/Contractor 
relationships,  there  continues  to  be  a 
concern  on  the  part  of  the  contractors  that 
this  assessment  will  be  used  as  a  “hammer” 
by  the  SPO.  In  addition,  we  made 
assurances  that  results  would  not  be  used  as 
an  Award  Fee  criteria.18  To  further  dispel 
contractor  concerns  and  in  keeping  with  the 
F-22  teaming  philosophy,  assessments  are 
conducted  across  companies  and  the  SPO. 
For  example,  the  Airframe  team  consists  of 
sub-IPTs  from  the  F-22  Systems  Program 
Office  (SPO),  Lockheed  Aeronautical 
Systems  Co.  (LASC),  Lockheed  Fortworth 
Co.  (LFWC)  and  Boeing  Military  Airplanes 
Division  (BMA);  therefore,  the  assessment 


takes  all  4  sites  into  consideration.  The 
Airframe  IPT  co-leads  at  the  SPO  and 
contractors  develop  their  assessments 
together  and  brief  them  together  at  PMRs. 
In  this  regard,  it  is  truly  a  team  assessment. 

2)  The  wording  of  the  template  was  chosen 
to  be  applicable  at  any  of  the  contractors/ 
SPO  and  at  any  level  of  the  F-22  Team. 
Therefore,  the  wording  is  necessarily 
general  and  not  specific  to  the  government. 
It  represents  a  “best  fit”  between  four 
diverse  contractors  and  the  SPO,  and  all  the 
tiers  of  management  within  each.  Some 
categories  and  descriptions  will  be  more 
applicable  to  individual  organizations  than 
others.  The  wording  is  meant  to  be  used  as 
a  guide  for  further  discussions  within  and 
between  IPTs.  Users  of  the  template  should 


STAGE  I 

STAGE  II 

STAGE  III 

STAGE  TV 

STAGE  N 

3.5  Bench  marking 
-  search  for  best 
practices  leading  to 
superior 
performance 

-  internal  company 
lessons  learned  used  to 
improve  processes 

-  knowledge  of  other 
companies;  best 
practices  is  increased 
and  used  to  improve 
processes 

-  industry  best 
practices  used  as 
benchmarks  for 
improvement  of 
processes 

-  most  processes 
have  benchmarks  to 
strive  for 

-  systematic  approach 
for  continuous 
improvement  through 
competitors  /  like 
processes  (e.g.,  process 
cyde  time,  defects  per 
unit  measure) 

3.6  Metrics 
Development  & 
Usage  -  use  of 
metrics  to  achieve 
continuous 
improvement 

-  not  enough  of  the  right 
metrics 

-  too  many  of  the  wrong 
metrics 

-  just  collecting  metric 
data 

-  metrics  used  to 

control  some 
processes 

-  metrics  data  drives 
improvements 

3.7  Risk 

Manaeement  -  how 
risks  are  identified 
and  mitigated 

-  ad  hoc  process  for 
identifying  some  risks 

-  systematic  approach 
in  identifying  risks 

-  business 

management  & 
systems  eng.  mgL 
develop  an 
approach  to  identify 
and  mitigate  risks 

-  teams  identify  and 
mitigate  risks  at  lowest 
level 

3.8  Customer 
Satisfaction 
Feedback  -  process 
for  obtaining  and 
using  customer 
feedback  to 
improve  processes 
&  products 
(customer  is  any 
party  that  receives 
the  result  of  a 
process) 

-  no  customer  feedback 
solicited 

-  individual  disciplines 
optimize  their  areas  using 
prior  experience 

-  ad  hoc  customer 
feedback  solicited 

-  limited  use  of 
feedback 

-  systematic 
feedback  process  in 
place 

-  feedback  on 
difficult 

improvement  areas 
ignored 

-  IPTs  establish 
channels  to 
continuously  gather 
feedback 

-  feedback  used  by 

IPTs  to  improve 
processes/product 

Figure  23  Reference  Level  Template  -  Category  3.0  "Processes"  (Cont.) 
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not  focus  on  the  specific  wording,  but  use  it 
as  a  framework  and  tailor  it  for  their  own 
applications. 

3)  The  assessments  should  not  be  “rolled 
up”  or  averaged  from  the  reference  level 
template  to  the  top  level  template. 
Similarly,  the  results  from  lower  tiers  should 
not  be  averaged  to  assess  a  higher  tier. 

4)  It  is  generally  assumed  that  farther  to  the 
right  on  the  template  is  better.  However, 


Catego 


Allocation  -  process 
for  distribution  of 
human  resources 


4.2  Training  and 


Education  -  training 
teaches  the  how  of 
the  job.  Education 
teaches  the  why  of 
the  job 

-  functionals  nurture 
&  broaden 
individual 
disciplines 


43  Recognition  - 


basis  for 

awards/recognition 


4.4  Performance 
Appraisals  -  how 
project  members  are 
rated  and  by  who 


43  Leaders  hi 


who  leads  the  team 
and  how  the  team  is 
led 


STAGE  I 

STAGE  II 

•  resources  tied  to 
specific  disciplines 
rather  than  products 

-  top  level 

management  allocates 
human  resources 
based  on  product 
development  needs 

-  training  emphasis  on 

specialties/functional 

skills 

-  group  dynamics  and 
conflict  management 
included  in 

curriculum 

-  limited  individual 
awards  based  on 
functional  knowledge 

-  members  want 
individual  awards 

-  individual  awards 

-  management 
rewards  "Lone 

Ranger”  heroes  for 
solving  short  term 
crises 

-  members  rated  by 
functional  chief  based 
on  individual 
functional 
performance 

-  members  rated  by 
functional  chief  based 
on  individual 
performance  on  team 

-  centralized, 
functional  lines 
control  activities 

-  management 
appoints  leader 

-  management- 
selected 

interdisciplinary  team 
leader 

there  may  be  cases  where,  an  organization 
does  not  want  to  progress  further  to  the 
right,  given  the  complexity  of  the  project 
and/or  the  resources  available.  For  example, 
a  team  with  a  relatively  simple  task  may  not 
need  the  most  advanced  methods  of  systems 
integration.  To  implement  them  may  be  too 
costly  or  time  consuming  for  the  task  at 
hand.  The  team  could  assess  themselves  at 
Stage  II  or  III  and,  looking  ahead,  decide 
they  have  progressed  as  far  as  is 


STAGE  III  STAGE  IV  \  STAGE  N 


-  team  members 
assess  resource  needs 
and  identify  and 
select  members 

-  functionals  provide 
motivated,  skilled, 
trained  resources  as 
candidates 


-  training  and 
education  based  on 
formal  group 
assessment  of  team 
performance 

-  training  &  education 
are  tailored  to  meet 
team  needs 


-  resources  allocated  to 
the  teams  by  functional 
organizations 

-  team  leader 
communicates  human 
resource  requirements 
to  functionals 


-  team  training  initiated, 
whole  teams  are  trained 
as  a  unit 

-  cross  functional 
specialty  awareness 
education 


-  team  allocates  award 
to  individuals 

-  "Hero"  status  is  team- 
determined 

-  award  based  on 
contribution  to  team 
goals/objectives 


-  team  leaders  rate 
member  on  team 
performance 

-  functionals  rate 
member  on  discipline 


-  team  award  based 
on  contributions  to 
long  term  program 
goals,  not  for  short 
term  sub  optimization 

-  "Heroes"  not 
rewarded  if  team  fails 

-  members  demand 
team  awards  versus 
individuals 


-  team  member  inputs 
considered  in 
performance  appraisal 


-  team  selected 
interdisciplinary  leader 


Figure  2.4  Reference  Level  Template  -  Category  4 


E3 


-  formal 

interdisciplinary 
leader  with  subject 
matter  leaders 
emerging  based  on 
ability  &  knowledge 
required  for  specific 
task 

-  all  team  members 
accept  accountability 
for  team  performance 


"Human  Resource  Management" 


economically  practical.  In  this  case  the 
template  is  also  valuable:  it  has  given  the 
team  a  method  to  assess  their  current  state, 
evaluate  possible  “next  steps”  that  could  be 
taken,  and  make  a  conscious  decision  to 
remain  in  their  current  state.  The  team  will 
not  waste  resources  pursuing  improvement 
just  for  improvement  sake.  However,  most 
teams  using  the  template  will  find 
improvements,  no  matter  how  small,  that 
can  be  made  to  increase  their  effectiveness. 

5)  There  should  be  no  stigma  attached  to 
assessments  at  lower  stages.  This  is  simply 
an  indication  of  the  maturity  level  of  IPD 
within  the  organization  and  the  distance  yet 
to  travel.  The  focus  should  be  on  making 
systematic  evaluations  and  developing 
action  plans  for  improvement.  It  is  also 
important  to  realize  that  teams  may  move  to 
a  lower  stage  of  the  template  over  time  due 
to  changing  program  conditions,  changes  in 
personnel,  or  implementation  of  new 
policies/processes. 

The  F-22  Team  IPD  Implementation 
Improvement  Process.  As  important  as  the 
template  itself  is  the  process  developed  to 
drive  continuous  improvement  of  IPD 
implementation.  Following  the  PMR  trial 
runs,  the  project  team  revised  the  template 
and  refined  the  implementation  process.  In 
developing  this  process,  the  project  team 
considered  these  factors: 

(1)  To  optimize  time  spent  by  the  IPTs, 
the  process  should  dovetail  with  normal 
operations  of  the  F-22  Team  and  there 
should  not  be  separate  reviews 


(2)  The  process  should  compliment  the 
F-22  Team  Plan19  (i.e.,  strategic 
planning)  process 

(3)  The  process  should  emphasize 
improvement  as  well  as  assessment 

The  team  briefed  its  recommendations  for 
changes  and  further  use  of  the  template  at 
the  November  1994  F-22  Team  Executive 
Offsite  (the  annual  strategic  planning 
meeting  of  senior  leaders  from  across  the 
F-22  Team).  The  offsite  participants 
accepted  the  recommendations  of  the  project 
team  to  continue  use  of  the  template  through 
1995.  As  shown  in  Figure  3.0,  the  process 
consists  of  the  following  elements: 

Annual  IPT  Assessment:  Each  Tier  I,  II  and 
limited  Tier  III  IPTs  use  the  template  to 
perform  an  annual  assessment.  The 
assessment  takes  place  in  September  or 
October,  timed  so  that  the  IPTs  can  develop 
action  plans  to  work  for  the  coming  year, 
and  identify  program-wide  IPD  issues  for 
the  F-22  Executive  Offsite.  The  IPTs  assess 
their  current  status  on  the  template  and  their 
desired  (goal)  status.  At  the  same  time,  they 
identify  obstacles  that  are  out  of  their 
control  that  block  their  progression  to  the 
next  stage.  These  obstacles  are  raised  as 
issues  to  the  next  higher  tier  and,  eventually, 
to  the  Executive  Offsite  for  resolution. 

Developing  Action  Plans:  The  IPTs  develop 
action  plans  to  help  move  them  further 
along  toward  full  IPD  implementation. 
Action  Plans  consist  of  the  action  needed, 
the  responsible  individuals,  the  planned 
completion  dates,  and  any  interim 
milestones. 
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PMR 


PMR 


PMR 


PMR 


PMR 


PMR 


r  r  r  7 


►[  ifTS  BRIEF  ASSESSMENTS,  STATUS  OF  ACTION  PLANS,  AND  BEST  PRACTICES  AT  PMR* 
-►f  IPTi  EXAMINE  BEST  PRACTICES  FOR  APPLICABILITY 


IFTi  CONDUCT 
ASSESSMENT 

-  DEVELOP 
ACTION  PLANS 
FOR  NEXT 
YEAR 

-  FEED 
RESULTS  AND 
ISSUES  TO 
EXEC  OFFSITE 


IPTs  WORK  ACTION  PLANS 


EXEC 


f  Review  results  of  IPT  a 


1-  Considers  progra»-widc  IPD  imea 
OFFSITEj  V  Developa  Iocm  artu  for  tnprowncat 
^  Sdi  direction  for  coatag  year 


YEARLY  TEAM  PLAN 


|  IPD  TEMPLATE  PROJECT  TEAM  MAINTAINS  /  IMPROVES  TOOL  AS  NEEDED 


> 

3> 


3> 


Figure  3.0  -  F-22  IPD  Implementation  Improvement  Process 


3> 


Sharing  Best  Practices:  To  help  the  flow  of 
information  across  IPTs,  each  IPT  identifies 
and  briefs  “Best  Practices”  (i.e.,  good  ideas, 
innovative  processes,  or  management 
techniques  that  may  be  applicable  to  other 
IPTs). 

Status  Progress  Reports:  IPTs  brief  their 
assessments  and  Action  Plan  status  at 
Program  Management  Reviews  which  are 
held  every  8  weeks. 

Template  Maintenance:  The  template 

project  team  maintains  and  updates  the 
template  as  needed  throughout  the  year 
based  on  the  development  of  new  stages  by 
the  IPTs.  In  addition,  the  project  team 
maintains  a  database  of  Best  Practices 
presented  by  the  IPTs  at  the  PMRs. 


the  first  iteration  of  the  IPD  Implementation 
Improvement  Plan  have  produced  the 
following  lessons  learned  from  the  IPTs 
across  the  F-22  Team: 

o  Understand  who  all  your  customers  are. 
Active  customer  involvement  is  essential. 
Traditional  roles  must  change. 

o  Top  level  leadership  (tier  1)  on  the  part  of 
both  government  and  contractor  must 
visibly  commit  to  an  IPD  continuous 
improvement  plan  for  it  to  work. 

o  IPT  training  is  essential  at  all  levels 
(Company/command,  program,  and  IPT 
member  levels). 

o  Management  cultural  changes  do  not 
occur  overnight  -  functionals  tend  to  resist 
IPT  arrangements.  IPT  leaders  must  have 


Results  to  Date.  The  past  three  years  of 
IPD  implementation  on  the  F-22  Team  and 
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authority  over  personnel  and  budget 
resources. 

o  IPTs  must  have  experienced  and 
empowered  members  to  fully  represent  their 
functional  discipline  (e.g.,  engineering, 
finance,  etc.).  Otherwise  the  product  will 
suffer  in  that  functional  discipline. 

o  The  “I”  in  IPT  can  easily  become 
“Independent”  vice  “Integrated”. 
Integration  across  the  IPTs  at  every  level  is 
absolutely  imperative. 

o  IPT  members  should  sit  together.  IPTs 
dispersed  at  various  sites  should  meet  on  a 
periodic  basis  .  An  integrated  network  of 
software  tools  is  extremely  useful  to 
improve  communication  between  dispersed 
IPTs. 

o  Set  team  goals/objectives  -  and  track 
them!  Ensure  all  team  members  participate 
in  decisions.  Develop  meaningful  team 
metrics. 

o  Put  the  right  people  in  the  right  job  at  the 
right  time.  Appoint  and  train  “LEADERS”. 
Replace  leaders  that  don’t/can’t  lead. 

o  IPTs  are  wary  of  criticism  by  top 
leadership  as  they  use  the  template  and 
identify  improvement  areas.  IPTs  need  the 
opportunity  to  improve. 

o  IPTs  were  convinced  of  the  usefulness  of 
the  IPD  Implementation  Improvement  Plan. 
It  provided  a  structured  review  of  IPT 
effectiveness  and  identified  possible  areas  of 
improvement. 

SUMMARY 

The  F-22  Team  will  continue  to  use  the  IPD 
Implementation  Improvement  Plan  as  they 
work  Action  Plans,  identify  Best  Practices 


and  perform  another  IPT  assessment  in  the 
Fall  of  1995.  Linkage  of  the  plan  to  the 
F-22  Team  (Strategic)  Plan  provides  a 
natural  cycle  for  review  and  replan  as 
necessary  at  the  annual  Executive  Offsites. 
Only  through  a  systematic  approach  of 
measurement  and  planning  such  as  this  can 
the  primary  goal  of  meeting  the  F-22’s 
mission  be  achieved. 
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13AFMC  Pamphlet  800-60  Integrated  Weapon 
System  Management  Implementation  Guide. 
Reference  Attachment  E.  “Integrated  Product 
Development  Guide”,  31  March  1993.,  p.  289 

14Report  on  the  DoD  Workshop  on  Concurrent 
Engineering,  Defense  Systems  Management 
College,  November  14-15, 1990;  December  17-21, 
1990;  and  January  28-29,  1991, 

15AFMC  Pamphlet  800-60  Integrated  Weapon 
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17First  Principles  of  Concurrent  Engineering  -  A 
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CALS/Concurrent  Engineering  Working  Group- 
Electronic  Systems,  12  May  1992. 

18The  F-22  Contract  is  a  cost-plus  Award  Fee 
Contract.  An  Award  Fee  is  paid  to  the  contractors 
every  six  months  based  on  a  subjective  evaluation  of 


their  performance  by  the  SPO.  The  Award  Fee 
criteria  are  written  and  agreed  upon  by  the  SPO  and 
the  contractors  every  six  months  and  form  the  basis 
of  the  evaluation. 

19In  a  concurrent  effort  during  1994,  the  F-22  Team 
developed  its  initial  Team  Plan,  the  strategic  plan 
for  the  F-22  Team.  This  plan  identifies  the  F-22 
Team’s  vision,  mission,  and  goals,  and  establishes 
focus  areas  to  be  worked  over  the  next  year. 
Participants  at  the  November  1994  F-22  Team 
Executive  Offsite  ratified  the  Team  Plan  and 
determined  the  focus  areas,  by  priority,  that  the  team 
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TAKING  THE  NEXT  STEP: 
IMPLEMENTING  ORGANIZATIONAL  CHANGE 


LtCol  Robert  K.  Saxer 
National  Defense  University 
Industrial  College  of  the  Armed  Forces 


There  is  nothing  more  difficult  to  take  in  hand,  more  perilous  to  conduct, 
or  more  uncertain  in  its  success  than  to  take  the  lead  in  the  introduction 
of  a  new  order  of  things. 

Niccolo  Machiavelli 1 


ABSTRACT 

To  survive  in  today’s  highly  competitive 
operating  environment,  organizations  must 
leam  how  to  effectively  implement  change. 
Historically,  this  has  proven  difficult, 
particularly  in  bureaucratic  organizations  like 
DOD.  Our  natural  tendency  is  to  focus  on 
analysis  and  prescription,  rather  than  to 
tackle  the  tougher  challenges  of 
"implementation."  This  article  addresses 
those  challenges,  their  underlying  causes, 
and  offers  some  thoughts  and  ideas  for 
consideration  and  use  throughout  the  entire 
change  process. 

INTRODUCTION 

When  the  Federal  Acquisition  Streamlining 
Act  of  1994  was  signed  into  law  this  past 
October,  it  was  widely  hailed  as  one  of  the 
most  important  steps  the  nation  could  take 
toward  reforming  defense  acquisition  and 
meeting  our  long-term  strategic  economic 


and  policy  objectives.  Lengthy  detailed 
military  specifications  and  acquisition 
procedures  will  now  give  way  to  broader 
performance  based  standards,  best 
commercial  practices,  and  electronic 
commerce,  along  with  the  promise  of  an 
expanded  supplier  base  and  hundreds  of 
millions  in  annual  savings.  After  years  of 
delays,  false  starts,  and  numerous  "Blue 
Ribbon"  studies,  procurement  reform  appears 
to  have  finally  come  to  the  Pentagon.  Or 
has  it? 

Intuitively  we  have  our  doubts.  We  know 
from  experience  that  simply  passing  new 
laws  and  shipping  copies  to  the  field,  does 
not  guarantee  a  new  order,  but  merely 
signals  the  beginning  of  the  more  difficult 
and  elusive  task  of  implementing 
organizational  change.  Change  that  can 
occur  only  through  effective  and  determined 
leadership,  clear  strategic  vision  and  goals, 
and  the  successful  management  and 
execution  of  well  thought  out 
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implementation  plans.  Change  which  moves 
us  beyond  fuzzy  top  level  guidance  written 
in  law  and  toward  a  more  specific  series  of 
actionable  steps  tied  directly  to 
organizational  processes,  individual 
responsibility,  and  measurable  results 
centered  success  criteria. 

ORGANIZATIONS  -  A  WAY  OF  LIFE 

Organizational  reform  movements  are 
certainly  not  new.  Ambitious  efforts  to 
enhance  everything  from  manufacturing 
processes  to  internal  communications  have 
been  tried,  and  continue  to  be  commonplace. 
Historically,  however,  organizational  change 
has  proven  a  tough  sell,  with  most  reform 
efforts  falling  far  short,  often  contributing 
little  or  nothing  to  bottom-line  performance.2 
We  shouldn’t  be  surprised.  Change 
competes  with  our  natural  preferences  for 
stability  and  control.  Collectively  whether 
we  work  in  government  or  industry  we 
reside  in  organizations  out  of  necessity, 
protective  structures  that  evolve  over  time  in 
response  to  personal  needs,  cultural  biases, 
and  group  norms.  "We  are  born,  raised,  and 
educated  in  organizations.  We  work  in  them 
and  rely  on  them  for  goods  and  services. 
Many  of  us  will  grow  old  and  die  in 
organizations.  We  have  built  so  many 
organizations  because  of  what  they  can  do 
for  us."3 

Ironically,  even  though  the  proliferation  of 
complex  organizations  has  made  almost 
every  human  activity  a  collective  one,  we 
instinctively  resent  the  forces  that  manage 
and  control  them.4  More  of  us  than  ever  are 
unwilling  to  subject  ourselves  to  an 
organization  or  the  discipline  of  a  trade, 
profession,  or  team.5  This  love-hate 
relationship  with  the  organizations  governing 
our  lives  highlights  the  virtues  and 
drawbacks  of  coordinated  activities,  the  wide 
chasm  between  managerial  intent  and 


organizational  accomplishment,  and  the 
human  element  that  must  embrace  change  in 
order  for  "implementation"  to  succeed.6 

TOWARD  IMPLEMENTATION 

"Successful  fundamental  change  [has]  three 
components:  the  analytic  component 

(focusing  on  the  deficiencies  to  be 
corrected),  the  prescriptive  component 
(setting  forth  the  objectives  of  the  change 
and  its  general  characteristics),  and  the 
strategic  component  (dealing  with  the 
strategy  and  tactics  for  the  process  of 
change)."7  All  of  us  are  familiar  with  the 
first  two:  diagnosis  and  prescription.  In  fact 
it  is  difficult  to  imagine  not  being  able  to 
find  someone  in  government  or  industry  with 
an  opinion  on  how  to  improve  current 
operations.  Unfortunately  recognizing 
problems  and  offering  suggestions  is  the 
easy  part.  The  change  process  gets 
significantly  more  difficult  when  a  strategy 
and  action  plan  for  "implementation"  are 
required. 

DOD’s  acquisition  system  is  a  classic  case 
in  point.  For  over  twenty  years  high-level 
commissions  have  offered  analysis  and 
prescriptions  for  defense  procurement 
reform,  however,  we  consistently  have  been 
unable  to  take  the  next  step  forward  toward 
implementing  change.8  Our  downfall  has 
been  an  inability  to  formulate  the  concrete 
measures  necessary  for  putting 
recommendations  into  practice  or  to  translate 
new  ideas  into  specific  actionable  steps  that 
could  be  carried  out.9  Although  some  may 
argue  policy  limitations  make  substantial 
change  all  but  impossible,  DOD’s  situation 
is  not  unique.  All  large  organizations 
encounter  a  like  set  of  personal,  professional, 
and  political  constraints  as  they  move 
through  the  change  process.  Congressional 
inquiries,  labor  disputes,  shareholder 
demands,  and  government  regulations,  along 
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with  competitive  market  forces  all  combine 
to  make  commercial  sector  reform  efforts 
equally  difficult.  And  ultimately,  every 
organization  once  it  completes  the  analytic 
and  prescriptive  phases  of  the  change 
process  faces  the  same  chronic  dilemma,  the 
successful  development,  management  and 
execution  of  the  strategic  and  tactical 
components  of  change,  well  thought  out 
implementation  plans. 

CULTURE  WARS 

Lack  of  a  well  defined  or  well  executed 
implementation  plan  is  not  an  unusual 
situation,  nor  is  it  confined  only  to 
government  operations.  "Many  [corporate] 
strategic  plans  die  because  of  lack  of 
implementation.  There  is  a  high  correlation 
between  the  failure  to  implement  changes 
and  the  lack  of  conscious  management  of  the 
transitional  process."10  Clearly, 
implementing  actionable  change  at  any  level 
within  an  organization  is  not  easy.  Change 
challenges  our  sense  of  normalcy,  bringing 
fear,  uncertainty,  and  loss.  "Ask.. .people 
why  change  is  hard,  and  sooner  or  later  - 
usually  sooner  -  you’ll  hear  the  word 
"culture."  Changing  corporate  culture,  that’s 
murder.  It  will  rise  up  and  smite  you."11 
This  is  particularly  true  when  you’re 
breaking  new  ground.  "By  definition  you 
have  different  values  from  your  management 
-  both  the  people  you  work  for  and  those 
who  work  with  you.  The  risk  of  failure  is 
great.  Snipers  and  cynics  may  be  waiting 
anywhere  -  higher  up  in  the  company, 
among  peers,  among  subordinates."12 

We  shouldn’t  be  surprised  by  institutional 
resistance.  As  a  workforce,  we  have  been 
culturally  programmed  to  avoid  failure,  risk, 
and  confrontation,  to  conform  to  established 
procedures,  and  to  operate  within  clearly 
drawn  lines  of  authority  and  responsibility. 
Venturing  outside  the  "white  lines"  to 


implement  change  is  especially  difficult, 
particularly  if  we  must  give  up  power  or 
control  over  our  own  lives  to  do  so.  And 
nowhere  is  change  avoidance  more  evident 
than  in  today’s  highly  charged  operating 
environment.  Explosive  political,  economic, 
technological,  and  societal  forces  are  driving 
dramatic  service  realignments,  mission 
expansion,  base  closure  decisions,  and 
civilian  workforce  downsizing.  Accepting 
change  or  giving  up  control  may  now  mean 
losing  your  job.  The  personal  stakes  are 
high. 

Even  senior  acquisition  leaders  who  have 
lead  past  procurement  reform  efforts  and 
have  always  shown  great  affinity  and 
enthusiasm  for  diagnosis  and  change,  in 
practice  find  it  too  difficult  to  focus  on 
"implementation."  Instead  they  preferred 
drifting  back  toward  a  continuing  analysis  of 
what  is  wrong  with  the  current  system.13 
Perhaps  this  explains  our  natural  inclination 
for  studying  problems  vice  solving  them. 
And  while  it  is  easier  to  allow  our  known 
cultural  and  organizational  biases  to  protect 
us  then  to  deal  with  them  head-on, 
understanding  both  the  positive  and  negative 
role  they  play  in  human  and  organizational 
behavior  patterns  is  key  to  effectively 
dealing  with  the  broader  challenges  of 
implementing  change. 

SETTING  THE  COURSE 

Although  corporate  culture  remains  both  an 
obstacle  and  asset  to  be  managed,  true 
organizational  change  can  only  be  successful 
if  it  is  actively  supported,  well 
communicated,  and  fully  understood  by  top 
management.  Top  management  must  be 
committed  and  lead.  Senior  leadership’s 
broad  perspective  and  corporate  view, 
authority  to  cross  functional  boundaries,  and 
power  to  motivate  employees  beyond 
marginal  gains  is  critical  to  success.14  This 
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is  not  to  say  employees  should  not  be  a 
large  part  of  the  change  process  or  that  they 
can  not  be  a  source  of  great  ideas  and 
magnificent  support.  On  the  contrary,  it  is  to 
emphasize  that  significant  organizational 
change  comes  from  engaged  leaders  with 
well  defined  agendas  and  clear  corporate 
visions  for  the  future.  "There  is  no  more 
powerful  engine  driving  an  organization 
toward  excellence  and  long-range  success 
than  an  attractive,  worthwhile,  and 
achievable  vision  of  the  future,  widely 
shared."15 

Beyond  setting  a  vision,  however,  effective 
leaders  must  also  show  the  way.  Followers 
want  their  leaders  to  take  them  on 
increasingly  inspirational  journeys  and  in 
fact  judge  them  on  their  ability  to  set  the 
course  and  steer  them  toward  it.16  Failure  by 
management  to  take-on  this  role  or 
assumptions  the  latest  gimmick  can  solve  all 
problems,  encourages  defeat  and  leaves  an 
organization  groping  for  direction.  "One 
[sure]  sign  that  a  company  is  Clueless  in 
Seattle,  or  wherever,  is  [when]  top 

management  starts  succumbing  to 
management  fads.. .and  [after]  several  rounds 
of  reengineering... [t]he  work  force  can 
become  jaded  in  the  process  and  suffer  what 
Peter  Scott-Morgan... calls  change  fatigue. 
The  eventual  result  is  often  widespread 
worker  resistance  to  new  initiatives,  which 
frustrates  management  further  and  can  spiral 
into  ever  more  draconian,  equally 

unsuccessful  efforts  to  motivate  the  troops."17 

Escaping  from  this  often  self-imposed  death 
spiral,  however,  is  difficult  until  we  know 
ourselves.  Usually  we  don’t.  We 

demonstrate  a  "great  reluctance  [for] 

recognizing]  just  how  fundamentally  flawed 
many  [of  our]  organizations  are.  This  is  a 
splendid  example  of  what  sociologists  call 
pluralistic  ignorance.  Everyone  is  aware  of 
the  same  thing,  but  because  people  think  it 


true  only  for  themselves,  all  pretend 
otherwise.  Refusal  to  face  the  deep  flaws  at 
the  core  of  organizations  deflects  attention 
from  the  real  problems  to  the  symptoms."18 
Moving  beyond  merely  treating  symptoms 
requires  visionary  leadership,  both  of  what  is 
possible  and  what  is  desirable.  There  is  no 
mystery  here.  Leaders  are  the  catalysts. 
Their  sense  of  urgency  and  campaign  for 
change  focuses  the  agenda,  breaks  the  status 
quo,  maximizes  employee  involvement,  and 
begins  the  real  work  of  organizational 
transformation. 

SHARPENING  THE  FOCUS 

Taking  the  next  steps,  however,  requires  a 
significant  shift  in  organizational  focus. 
Today  much  of  the  work  done  in  large 
organizations  exists  solely  to  support  internal 
operations  rather  than  to  deliver  goods  and 
services  to  meet  customer’s  needs.  Bloated 
bureaucratic  layers  attest  to  management 
excesses  "...with  each  layer  having  an 
opportunity  to  reject,  deny,  modify,  filter, 
eliminate,  reduce,  stretch,  or  otherwise  retard 
every  suggestion  that  has  the  audacity  to 
seek  to  wind  its  way  through  the  labyrinth¬ 
like  approval  process  adopted  by  most  large 
organizations."19  Moving  beyond  these 
bureaucratic  "activity  centered"  programs 
and  toward  "results  driven"  processes  that 
contribute  directly  to  our  organization’s 
bottomline  must  be  our  objective.20  For  too 
long  we  have  defined  success  not  by  how 
well  we  have  met  or  exceeded  our  customers 
definition  of  quality,  but  by  whether  we 
answer  the  day-to-day  demands  our 
organizations  have  imposed  on  us.  And 
sadly,  we  have  been  subtlety  indoctrinated 
into  an  organizational  mindset  which  equates 
100%  attendance  at  quality  training  courses 
with  success  and  suggestions  that  any 
attempt  at  implementing  change  must  be 
slow  and  painful.21 
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Breaking  this  mindset  and  sharpening  our 
focus  on  results  driven  improvements  can  be 
a  powerful  tool  for  implementing  lasting 
change.  By  identifying  critical  areas  where 
improvements  are  urgently  needed  and  then 
ensuring  those  who  study  and  recommend 
solutions  are  also  held  accountable  for 
producing  measurable  results  in  a  methodical 
step-by-step  fashion,  dynamic  change  will 
occur  and  productivity  will  increase. 
Employees  need  to  experience  both  the 
responsibility  and  success  that  come  from 
the  results  of  their  work.  Success  breeds 
confidence,  hones  skills  necessary  for 
continued  gains,  and  provides  a  learning 
environment  for  teaching  employees  to  deal 
effectively  with  the  competing  needs  of  all 
involved.  This  methodology  of  tying 
individual  responsibility  to  actionable  steps 
and  specific  results  challenges  our 
conventional  bureaucratic  approach  to 
problem  solving,  reinforces,  energizes  and 
rewards  performance,  forces  a  fundamental 
shift  in  organizational  focus,  and  places 
increased  discipline  and  thoughtfulness  into 
the  entire  corporate  planning  process. 

STRATEGIC  PLANNING 

Strategic  planing  is  perhaps  the  most  basic 
of  all  corporate  leadership  functions  and  the 
primary  building  block  for  successfully 
implementing  change.  During  the  change 
process,  good  planning  encompasses  a  wide 
range  of  functions,  including  everything 
from  setting  a  strategic  focus,  establishing 
organizational  goals,  and  allocating 
resources,  to  defining  success  criteria, 
scheduling  timelines,  and  determining 
individual  responsibilities.  Flexibility  is 
critical,  as  each  area  is  highly  dependant  on 
the  other  and  each  contributes  to  the  overall 
success  or  failure  of  the  planning  effort. 

Too  often,  however,  the  planning  process 
breaks  down.  In  our  zeal  to  tackle  change. 


our  implementation  plans  are  not  well 
thought  out  and  are  typically  managed 
through  the  normal  operational  hierarchy 
rather  than  through  a  dedicated  change 
management  resource.22  As  a  result, 
communication  suffers,  momentum  is  lost, 
and  change  stalls.  A  dedicated  change 
agent,  a  well  defined  implementation  plan, 
and  a  correctly  aligned  organizational 
support  structure  can  eliminate  these 
problems,  while  acting  as  a  facilitator  and 
conduit  for  feedback  by  providing  avenues 
for  timely  action  and  quick  responses  to 
problems  or  questions  as  they  arise.  They 
also  help  the  organization  institutionalize 
change.  "It  has  long  been  recognized  that 
organizations  have  great  power  to  shape 
behavior,  not  so  much  by  forcing  it  as  by 
encouraging  it.  Organizations  make  some 
things  easier  and  some  things  harder,  thus 
making  the  former  more  likely  and  the  later 
less  likely.  This  is  the  work  not  simply  of 
culture  -  something  in  people’s  heads  -  but 
rather  the  formal  aspects  of  the  organization, 
such  as  its  distribution  of  roles  and 
responsibilities,  people’s  authority  to  commit 
resources,  existing  budgets  procedures,  the 
physical  or  geographical  arrangement  of  its 
space  and  facilities,  differences  in 
information  access  and  availability,  and 
reward  and  recognition  systems."23  Fully 
integrated  organizations  and  dedicated 
planning  processes  build  trust,  teamwork, 
cohesion,  and  accountability,  while  further 
encouraging  the  continuous  shift  in  human 
behavior  being  brought  on  by  organizational 
change. 

LETTING  GO 

While  good  organizational  planning 
arrangements  can  make  or  break  any 
implementation  effort,  true  success  is  usually 
tied  to  a  managers  ability  to  let  go.  "The 
introduction  of  a  new  order  often  requires  a 
dramatic  shift  in  leadership  style  and  a 
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loosening  of  the  reins  of  control,  as  decision 
making  is  moved  closer  to  those  who  deal 
with  customers."24  Decentralization  does  not 
mean  that  organizations  must  approach 
anarchy,  even  though  managers  may  fear 
any  change  to  the  status  quo  threatens  their 
level  of  power,  authority,  and  influence 
within  an  organization.25  On  the  contrary,  it 
encourages  ownership,  reduces  risk,  and 
works  to  lessen  "...the  even  greater  danger 
that  errors  will  be  made  by  decision  makers 
both  unfamiliar  with  the  details  and  having 
insufficient  time  for  careful  consideration  of 
the  implications  of  detailed  decisions."26 

"Our  present  policy  seems  to  be  aimed 
primarily  at  not  letting  anything  go  wrong  - 
but  [often  has]  the  unintended  consequence 
of  precluding  us  from  letting  anything  go 
right.  Lamennais’s  apothegm  applies: 
Centralization  breeds  apoplexy  at  the  middle 
and  anemia  at  the  extremities.  The  key  to 
effective  management  of  the  defense 
acquisition  process  is  really  very  simple: 
employ  capable  people,  establish  clear  goals, 
monitor  results,  and  get  out  of  the  way."27 
You  can  expect  that  once  this  occurs, 
organizations  will  move  closer  to  their 
customers,  communication  will  improve,  and 
employees  will  understand  the  impact  of 
their  actions  and  be  able  to  respond  to 
customer  needs  faster  and  more 
perceptively.28  The  end  result  is  employees 
become  more  productive,  take  on  more 
responsibility,  and  typically  become  willing 
partners  for  implementing  change. 

CREATING  VALUE 

As  partners  throughout  the  change  process, 
it  is  also  important  employees  feel  they  will 
be  well  trained,  empowered,  and  have  a 
voice.  Open  communication  channels,  active 
organizational  involvement,  and  continuous 
training  programs  certainly  help,  but 
management  commitment  to  their  needs  is 


critical.  "One  of  the  fundamental  problems 
in  organizations,  including  families,  is  that 
people  are  not  committed  to  the 
determinations  of  other  people  for  their  lives. 
They  simply  don’t  buy  in."29  "Without 
involvement,  there  is  no  commitment.  Mark 
it  down,  asterisk  it,  circle  it,  underline  it. 
No  involvement,  no  commitment."30 

Building  employee  commitment  requires 
most  organizations  to  rethink  their  training 
methods  and  learning  processes.  "[V]ery 
few  organizations  have  a  conscious  strategy 
to  implement  [change]  in  a  learning  mode. 
Normally,  change  is  measured  only  in  terms 
of  results;  rarely  is  there  any  measure  of  the 
process  of  improvement."31  Through  the 
establishment  of  measurable  success  criteria, 
process  flows,  and  individual  "data" 
ownership,  loyalty  can  be  built.  Even  the 
best  people  will  produce  poor  results  if  they 
are  not  trained  to  understand  how  their 
actions  influence  corporate  performance. 
And  the  potential  for  performance 
improvements  is  unlimited  since  we  know 
that  "...people  genuinely  strive  to  produce 
what  they  intend.  They  value  acting 
competently.  Their  self-esteem  is  intimately 
tied  up  with  behaving  consistently  and 
performing  effectively.  Companies  can  use 
these  universal  human  tendencies  to  teach 
people  how  to  reason  in  a  new  way  -  in 
effect,  to  change  the  master  programs  in 
their  heads  and  thus  reshape  their 
behavior."32  Ultimately,  well  trained  and 
educated  employees  represent  the  most  basic 
element  for  ensuring  the  viability  and 
success  of  organizations,  and  their  long  term 
value  becomes  even  more  important  as 
"...[breakthroughs  in  technology  continue  to 
enhance  [our]  efforts  to  dismantle  the 
traditional,  hierarchical  corporate  structure 
and  to  realize  the  potential  of  a  networked, 
nimble,  customer-responsive  organization."33 
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CLOSING  THE  LOOP 

Responding  to  customer  needs  is  clearly  the 
number  one  reason  why  organizations 
change  or  consider  change.  Strategic  vision, 
senior  leadership,  corporate  planning, 
employee  empowerment,  and  individual 
training  all  play  active  roles  in  supporting 
the  implementation  process,  but  each  is 
highly  dependant  upon  open  communication, 
continuous  performance  feedback,  and 
measurable  results  for  determining  whether 
change  is  successful.  Successful 
organizations  value  feedback  from  any 
source  or  in  any  form.  Without  it,  they  can 
not  function  or  survive.  Every  employee 
actively  communicating  provides  an  insight 
and  perspective  necessary  to  set  and  run 
operations,  collect  data,  adjust  systems, 
correct  mistakes,  and  keep  customers 
informed.  This  is  especially  true  now  in  our 
information  society  where  for  the  first  time 
in  civilization  the  name  of  the  game  is 
people  interacting  with  other  people.34 

This  is^  particularly  true  in  a  service 
organization  like  DOD,  where  "...internal 
communication  among  employees  is  often 
more  important  than  the  more  visible 
external  communication  with  customers."35 
Clear  communication  of  job  expectations, 
joint  goal  setting,  and  measurement  and 
appraisal  of  results  help  define  employee 
performance  and  the  value  of  their 
productivity.  It  also  helps  to  close-the-  loop 
on  the  overall  problem-solving  process  by 
evaluating  results.36  "The  importance  of  [a] 
closed-loop  approach  to  solving  problems 
becomes  evident  when  you  recognize  that 
circumstances  change,  situations  change, 
people  change,  and  your  preferences  and 
values  change.  You  must  adapt  your 
solutions  to  the  changing  times,  and  this 
means  occasionally  rethinking  the  solution 
entirely."37  Constant  reevaluation  of  the 
entire  organizational  change  process  and  a 


continuous  reminder  that  what  is  most 
important  about  any  event  is  not  what 
happened,  but  what  it  means  must  be  our 
standard  operating  approach.38  By  collecting 
valid  data,  analyzing  it  carefully,  and 
constantly  testing  the  inferences  drawn  from 
it;  we  can  identify  not  only  those  activities 
which  add  the  most  value  and  improve  our 
ability  to  distinguish  between  routine  and 
creative  tasks,  but  we  can  see  the  historical 
results  of  our  entire  change  process  and  seek 
ways  to  improve  it.39 

LOOKING  FORWARD 

As  DOD  approaches  the  next  century,  we 
will  continue  to  be  buffeted  and  shaped  by 
many  powerful  forces.  How  well  we 
respond,  and  how  well  we  implement  change 
will  dictate  whether  we  can  break  from  the 
past  and  move  forward.  Over  the  years  we 
have  become  victims  of  our  own  success, 
trapped  by  earlier  achievements. 
Unfortunately  the  very  factors  that  produced 
yesterday’s  victories  may  now  create 
tomorrow’s  losses.  We  need  to  change.  But 
needing  change  doesn’t  make  it  happen, 
strong  leadership,  clear  vision,  and 
executable  implementation  plans  do.  Our 
future  depends  on  them.  Organizational 
survival  is  compelling  all  but  the  most  brain- 
dead  organizations  to  reinvent  themselves. 
Innovative  organizations  are  slashing  costs, 
cutting  cycle  time,  boosting  quality  and 
accelerating  responsiveness.  To  make  it  all 
work,  centralized  hierarchies  have  been 
flattened,  communication  improved,  and 
people  have  stopped  taking  orders  from 
bosses  and  started  taking  responsibility  for 
results.40  DOD  must  do  the  same.  And  as 
the  pace  of  change  accelerates,  we  must 
move  to  stay  ahead  of  it,  always  looking 
forward  for  better  ways  to  evolve  and  grow  - 
to  innovate  and  implement.  Organizations 
that  fail  to  keep  pace  will  not  survive. 
Those  that  do  embrace  change  must  be  as 


Admiral  Owens  suggests  "...unafraid  of 
asking  innovative  questions  and  not  afraid  of 
the  consequences."41  They  will  flourish, 
doing  so  by  implementing  meaningful 
organization  change. 
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ABSTRACT 

The  Department  of  Defense  (DOD) 
acquisition  system  is  frequently  criticized 
for  its  rigid  compliance  to  an  expensive, 
antiquated,  and  bureaucratic  process.  This 
process  has  been  well  documented  regarding 
its  inability  to  delivery  current  technology  to 
the  battlefield  in  a  timely  manner.  This 
problem  is  especially  acute  in  the  military 
communications  area  where  the  rapid  pace 
of  technology  results  in  design  obsolescence 
within  18  months.  In  October  1993,  the 
Department  of  the  Army  and  the  Army’s 
Combat  Development  community 
challenged  the  acquisition  community  to 
procure  and  field  a  prototype  combat 
communications  system,  the  AN/TSC-143, 
Super  High  Frequency  (SHF)  Tri-Band 
Tactical  Satellite  Terminal  (Tri-Band)  to  an 
operational  unit  in  less  than  one  year.  The 
Program  Executive  Officer  for 
Communications  Systems  (PEO  COMM) 
and  his  Project  Manager  for  Satellite 
Communications  Systems  (PM  SATCOM) 
successfully  responded  to  the  challenge  in  an 
era  before  the  Secretary  of  Defense,  William 
Perry  issued  his  Memorandum, 
“Specifications  and  Standards-A  New  Way 
of  Doing  Business,”  in  June  1994.  In  fact, 
many  of  the  bold  initiatives  outlined  by  the 
Secretary  can  also  be  found  in  reviewing  the 
innovations  implemented  by  the  Tri-Band 
program  team. 


AN/TSC-143,  TRI-BAND  TACTICAL 
SATELLITE  TERMINAL 


The  Tri-Band  procurement  is  a  prime 
example  of  how  the  acquisition  process  can 
be  streamlined  and  serves  as  a  standard  by 
which  the  user  community  will  measure  the 
performance  of  the  acquisition  community 
in  the  future.  The  normal  Procurement 
Administrative  Lead  Time  (PALT),  defined 
as  beginning  with  the  submission  of  an 
approved  Procurement  Data  Package  (PDP) 
by  the  PM,  until  contract  award  is  estimated 
by  the  United  States  Army  Communications 
Electronics  Command  (CECOM)  to  be 
approximately  234  days  for  a  best  value 
source  selection  of  this  dollar  value. 
However,  using  a  dedicated,  focused  "Team 
Fort  Monmouth"  approach,  a  select,  highly 
motivated  team  achieved  a  contract  award  in 
only  72  days,  a  savings  of  162  days  or  70%. 
The  total  time  from  the  initial  phone  call  to 
start  work  on  October  22,  1994,  until 
contract  award  was  only  154  days. 

The  objective  of  this  paper  is  to  provide  the 
reader  with  an  illustrative  example  of  how 
acquisition  streamlining  can  be  achieved 
using  a  focused  team  approach. 
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BACKGROUND 

During  DESERT  SHIELD/DESERT 
STORM,  U.S.  forces  in  Saudi  Arabia 
established  the  largest  most  complex  tactical 
satellite  network  in  history  using  the  DOD 
owned  and  operated  satellites  that  comprise 
the  Defense  Satellite  Communications 
System  (DSCS).  However,  the  DSCS  could 
not  provide  enough  capacity  to  support  the 
deployed  forces’  huge  and  ever  expanding 
demands  for  data  intensive  intelligence, 
logistics,  and  administrative  information  in 
addition  to  higher  priority  command  and 
control  communications.  As  a  result, 
commercial  satellite  capacity  and  ground 
satellite  terminals  had  to  be  leased  from 
commercial  vendors  and  operated  by 
contract  personnel  at  significant  expense  to 
DOD. 

Numerous  After  Action  Reports  reiterated 
the  need  for  military  tactical  ground 
terminals  capable  of  accessing  commercial 
satellites  in  order  to  augment  the  overloaded 
DSCS.  Also,  a  series  of  General  Accounting 
Office  (GAO)  reports  criticized  DOD  for  not 
using  more  commercial  satellite 
communications  in  support  of  day  to  day 
peacetime  operations,  as  well  as  in  support 
of  combat  operations.  However,  DOD  can 
neither  rely  on  commercial  vendors  of 
satellite  terminals  to  always  be  willing  to 
deploy  their  personnel  on  the  battlefield  nor 
to  remain  in  place  should  the  potential  for 
injury  or  capture  increase  to  a  level  the 
individual  may  deem  unacceptable.  During 
the  Gulf  War,  one  contractor  deserted  his 
equipment  the  same  hour  the  air  war  began, 
forcing  military  personnel  to  take  over  their 
equipment  to  maintain  communications. 

In  the  summer  of  1993,  the  Army  attempted 
to  fund  the  procurement  of  SHF  tactical 
satellite  terminals  capable  of  accessing  not 


only  the  DSCS  system  in  the  X  band  (7900- 
8400  Mhz),  but  also  the  commercial  Ku  (14- 
14.5  Ghz)  and  C  bands,  (5850-6425  Mhz) 
i.e.  Tri-Band  capable,  as  part  of  the 
Commercial  Space  Package.  The  United 
States  Army  Space  Command 
(USARSPACE)  proposed  that  they  be 
responsible  for  the  procurement  much  to  the 
dismay  of  the  acquisition  community. 

At  a  follow-on  meeting  in  September, 
USARSPACE  lobbied  for  an  increase  in 
staffing  of  13  personnel  to  conduct  the 
acquisition.  The  PEO  for  Communications 
Systems,  BG  David  Gust  assured  the 
community  that  he  could  procure  these 
systems  faster  than  any  non-acquisition 
oriented  organization  and  would  not  require 
additional  manpower.  Based  on  these 
assurances,  the  decision  was  made  to  assign 
the  mission  to  PM  SATCOM.  However, 
attempts  to  secure  funding  under  the 
Commercial  Space  Package  were 
unsuccessful  and  further  activity  was 
suspended. 

MISSION  IMPOSSIBLE 

On  October  22,  1993,  DA  unexpectedly 
directed  PM  SATCOM  to  begin  planning  for 
the  procurement  of  6  prototype  Tri-Band 
SHF  Tactical  Satellite  Terminals  using 
reprogrammed  funds,  not  to  exceed  $9.9 
million.  PM  SATCOM  was  also  directed  to 
procure  the  terminals  on  a  Non- 
Developmental  Item  (NDI)/Commercial- 
Off-The  Shelf  (COTS)  basis  for  operational 
use  by  the  Power  Projection  for  the  Army 
Command,  Control,  and  Communications 
(Power  PAC3)  Company. 

The  Power  PAC3  Company  was  scheduled 
for  activation  in  June  1994.  These  prototype 
terminals  were  to  be  capable  of  providing  a 
complete  communications  package  capable 
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of  providing  initial  Command,  Control,  and 
Communications  (C3)  to  a  deployed  Army 
Forces  headquarters  (ARFOR)  and  five 
liaison  teams.  DA  stipulated  that  this  be  a 
one  time  procurement  with  a  maximum 
quantity  of  6  terminals.  The  possibility  of 
including  options  for  additional  quantities 
was  discussed,  but  was  disapproved  by 
DCS  OPS. 

Although  the  schedule  dictated  an 
NDI/COTS  procurement,  The  United  States 
Army  Signal  School  demanded  that  these 
state-of-the-art  Tri-Band  terminals  be  fully 
capable  of  operating  under  battlefield 
conditions  using  military  operators.  Their 
requirements  were  very  ambitious  and 
presented  a  significant  challenge  to  industry. 
The  terminals  were  to  have  the  capability  of 
rapidly  switching  between  SHF  bands  under 
field  conditions  using  military  operators  and 
be  mounted  on  a  single  Heavy  High 
Mobility  Multi-Purpose  Wheeled  Vehicle 
(HHMMWV)  to  include  the  antenna,  an  on¬ 
board  generator,  and  an  integrated  digital 
switch. 

To  facilitate  rapid  force  projection,  a  major 
terminal  design  requirement  was  the 
capability  of  being  transportable  by  a  single 
C-130  aircraft.  At  the  time,  six  C-130’s 
were  required  to  deploy  a  similar  C3 
capability  that  still  would  not  provide  access 
to  commercial  satellites. 

The  Tri-Band  Terminal  was  also  required  to 
provide  a  minimum  data  rate  of  512  Kbps  in 
C  Band,  2  Mbps  in  X  Band  and  8  Mbps  in 
Ku  Band,  a  vast  increase  in  capability  over 
current  tactical  systems.  Additional  required 
capabilities  were  for  hub-spoke  operations 
and  backwards  compatibility  with  the 
current  family  of  Ground  Mobile  Forces 
(GMF)  tactical  satellite  terminals  (AN/TSC- 
85/93/94/100). 


The  challenge  to  both  the  PM  and  to 
industry  was  to  deliver  this  combat  ready 
“prototype”  satellite  communications  system 
to  the  unit  by  September  30,  1994. 

THE  TEAM  CONCEPT 

PM  SATCOM  implemented  several  key 
innovative  initiatives  at  the  outset  of  the 
program.  On  October  25,  1993,  the  PM 
established  a  Special  Project  Office  staffed 
with  existing  CECOM  matrix  and  PM  core 
personnel  tasked  to  provide  support  to  the 
acquisition  of  the  Tri-Bands.  This  action 
formed  a  dedicated,  focused  team  with  a 
common  goal  of  delivering  a  quality  product 
to  the  soldier  in  record  time.  The  PM  gave 
them  "carte  blanche"  to  make  it  happen, 
gave  them  good  management  support,  then 
got  out  of  their  way.  The  team  included 
members  of  the  functional  elements  of 
CECOM  that  included  the  Research, 
Development,  Engineering  Center  (RDEC) 
and  the  Command,  Control, 

Communications,  and  Intelligence  (C3I) 
Acquisition  Centers  and  the  Logistics  and 
Legal  Directorates.  No  additional  personnel 
were  added  to  organizations’  manning 
documents  for  this  procurement. 

The  team  avoided  a  "business  as  usual" 
approach.  From  the  beginning,  the  team 
adopted  a  zero  based  requirements 
philosophy  in  dealing  with  the  functional 
support  matrix.  Each  time  the  functional 
expert  requested  a  Contract  Data 
Requirement  List  (CDRL)  item  or  demanded 
an  action,  a  document,  or  the  incorporation 
of  a  military  specification  (MILSPEC)  or 
military  standard  (MILSTD)  that  didn't  make 
sense,  the  requester  was  challenged  to 
support  its  necessity.  "Boilerplate"  input  to 
the  specification  or  Statement  of  Work 
(SOW)  provided  by  the  matrix  was 
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drastically  tailored  to  fit  the  program  or 
eliminated  altogether 

A  key  element  for  successful  teaming  in 
matrix  support  organizations  is  unity  of 
command.  CECOM  management  deserves 
special  recognition  in  giving  the  PM  full 
control  of  the  matrix  personnel  on  the  team. 
Despite  the  fact  that  much  hard  work 
occurred  over  the  holiday  period,  the  PM 
was  able  to  delay  leaves  and  extend  duty 
hours  as  required  to  maintain  the  schedule. 

Essential  to  the  team  concept  was  getting  the 
Combat  Developer  and  the  ultimate  user  on 
board  as  members  of  the  team.  In  early 
November,  1993,  key  members  of  the  team 
conducted  a  three  day  review  of  the  first 
draft  specification  at  Fort  Gordon,  Georgia. 
During  this  review,  both  the  representatives 
from  the  combat  development  community 
and  soldiers  from  the  gaining  unit  were  able 
to  interface  directly  with  the  material 
developer  and  provide  comment  to  the 
specification.  Early  and  continuous 
involvement  with  these  representatives 
during  specification  development,  source 
selection,  and  testing  was  key  in  developing 
a  sense  of  trust  and  confidence  in  the  PM. 
This  made  the  tough  job  of  conducting  cost- 
performance  tradeoffs  in  light  of  cost  and 
schedule  constraints  much  easier. 

An  equally  important  member  of  the  team 
was  industry.  Industry  was  brought  into  the 
program  early  by  a  Commerce  Business 
Daily  (CBD)  announcement  on  November 
10,  1993.  The  CBD  announced  a  Briefing  to 
Industry  on  November  22,  1993  and 
described  the  program.  During  that  briefing, 
industry  was  provided  a  copy  of  the  draft 
specification  and  requested  to  provide 
comments.  In  addition,  one  on  one 
discussions  were  held  the  following  day  with 
every  prospective  offeror  desiring  to  meet 


with  the  government  team.  Many 
productive  comments  and  suggestions  were 
obtained  during  both  these  sessions. 

Industry  was  kept  informed  throughout  the 
PDP  preparation  through  a  continuous 
dialog  with  the  team  via  an  established 
Electronic  Bulletin  Board  (EBB)  system.  A 
Pre-Solicitation  conference  was  also  held  in 
January  1994  just  prior  to  RFP  release  to 
give  potential  offerors  one  final  chance  to 
ask  questions  and  again  have  one  on  one 
discussions. 

The  team  was  goal  oriented,  not  functionally 
oriented.  If  an  action  didn't  support  rapid 
delivery,  but  only  satisfied  an  administrative 
or  bureaucratic  requirement,  the  action  was 
modified  or  deleted.  The  PM  was 
challenged  by  the  functional  elements  of 
CECOM  demanding  normal  data 
requirements  and  adhering  to  policies  more 
suitable  for  developmental  programs.  In 
each  case,  the  PM  insisted  the  team  ask 
themselves  four  questions  before  reaching  a 
decision: 

Does  it  make  good,  common 
business  sense? 

Is  it  legal  and  ethical? 

Is  the  functional  requirement 
consistent  with  the  intent  of  the  program? 

Is  the  PM  willing  to  take  the  risk  and 
be  held  accountable  for  the  decision? 

Problems  were  worked  in  parallel  not  in 
serial.  Although  the  schedule  demanded 
immediate  solutions  to  produce  a 
Procurement  Data  Package  and  award  a 
contract,  the  team  managed  to  look  beyond 
that  and  simultaneously  initiated  other 
critical  actions.  The  team  was  required  to 
procure  GFE,  modify  the  HHMMWVs  with 
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the  400  ampere  alternator  kits,  and 
coordinate  for  operational  testing. 

Teaming  improved  productivity,  enhanced 
the  quality  of  the  product,  provided  better 
customer  support,  and  saved  time. 

Normally,  using  the  “business  as  usual 
approach,’1  the  PM  would  prepare  a  PDP, 
which  would  then  undergo  an  arduous 
approval  cycle  that  included  a  System  Data 
Review  Board,  a  legal  review,  and  a 
contracting  review.  Instead,  the  individuals 
from  these  functional  areas  joined  the  team 
and  actively  participated  in  the  development 
of  the  PDP  and  approved  it  as  it  was 
prepared.  As  a  result,  the  approved  PDP 
was  presented  to  the  CECOM  contracting 
activity  in  on  January  11,  1994.  Because  the 
highly  motivated,  innovative  Contracting 
Officer  assigned  to  the  team  from  the  first 
day  thoroughly  understood  the  program  and 
had  “done  his  homework,”  the  Contracting 
Officer  was  able  to  release  the  formal 
Request  For  Proposal  (RFP)  the  next  day. 

PULLING  OUT  ALL  THE  STOPS 

Key  to  the  success  in  this  program  was  the 
high  level  management  interest  and  support 
both  within  the  Combat  Development  and 
acquisition  community.  Throughout  the 
program,  key  managers  in  CECOM  and  the 
PM  were  willing  to  remove  non-performing 
personnel  who  were  unwilling  to  conform  to 
a  new  way  of  doing  business.  Senior 
executives  were  willing  to  coach  the  team  in 
the  art  of  best  value  acquisitions  of 
NDI/COTS  equipment. 

Another  significant  aspect  was  the 
willingness  of  our  acquisition  leaders  to 
stand  up  to  pressures  from  the  user 
community  who  were  indulging  in 
“requirement  creep”.  Throughout  the 
procurement,  and  even  up  to  the  day  before 


the  release  of  the  formal  RFP,  users, 
industry  and  consultants  demanded  last 
minute  changes  to  the  requirements.  These 
changes  would  have,  as  a  minimum,  caused 
delays  in  the  program  and  in  some  cases 
restricted  competition.  However,  personal 
involvement  by  both  the  PEO  and  the  CG 
CECOM  kept  the  procurement  on  track. 

The  team  implemented  several  initiatives  to 
tailor  this  acquisition  and  to  streamline  the 
development  of  the  PDP.  The  PDP  used  a 
functional  or  performance  based 
specification  that  allowed  the  contractor  to 
use  his  initiative  on  how  best  to  satisfy  the 
requirement.  Although  the  team  wrote  the 
first  draft  specification  to  reflect  the  ideal 
requirements  expressed  by  the  user,  it  was 
tailored  in  coordination  with  the  user  to 
reflect  what  really  was  achievable  in  the 
time  available.  Industry  comments  and  a 
good  market  analysis  were  key  in  convincing 
the  user  to  re-look  his  opeerational  needs 
and  to  establish  his  absolute  minimum 
requirements.  An  Army  Signal  School  two 
page  memorandum  served  as  the 
requirement  document  in  the  absence  of  a 
Mission  Needs  Statement  (MNS)  and 
Operational  Requirements  Document 
(ORD). 

The  Statement  of  Work  (SOW)  was  limited 
to  the  absolute  minimum  essential 
requirements  that  could  be  satisfied  by 
industry  in  the  short  time  available.  Many 
of  the  normal  logistics,  training  and  testing 
deliverables  were  tailored  to  meet  the  unique 
requirements  of  the  program  or  deleted 
entirely.  As  a  result,  the  number  of  data 
items  was  reduced  from  41  to  1 1  during 
development  of  the  PDP. 

The  key  streamlining  tool  was  the  use  of  the 
Electronic  Bulletin  Board  (EBB)  to  establish 
instantaneous  communications  with  industry 
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and  other  government  agencies.  Each 
revision  of  the  specification  and  SOW  was 
placed  on  the  EBB  for  industry  comment. 
These  comments  were  received  and 
responded  to  by  the  PM  via  the  EBB  within 
24  hours  of  receipt.  It  is  particularly 
noteworthy  that  the  Request  for  Proposal 
(RFP)  was  a  "paperless  solicitation,"  issued 
electronically  by  CECOM  C3I  Acquisition 
Center,  only  24  hours  after  receipt  of  the 
PDP  from  the  PM. 

CORRALLING  THE  TEST  COMMUNITY 

Although  these  Tri-Band  terminals  would 
mostly  consist  of  NDI  and  COTS 
components,  they  were  required  to  interface 
with  existing  tactical  communications 
systems,  DOD  strategic  systems,  and 
commercial  systems.  As  a  result,  there  was 
an  extensive  need  for  interoperability  and 
performance  testing.  However,  it  was 
obvious  that  time  and  money  did  not  allow 
the  test  community  to  conduct  business  as 
usual. 

Four  essential  test  events  established  by  the 
PM  were  an  In-Plant  Acceptance  Test,  the 
DSCS  and  Commercial  Satellite 
Certification,  a  Final  Installation  Acceptance 
Test  conducted  at  the  unit's  location,  and  an 
OPTEC  sanctioned  Customer  Test.  The 
Test  and  Evaluation  Control  Office  at  Fort 
Gordon,  Georgia  would  plan  and  supervise 
the  test  in  a  Field  Training  Exercise 
supported  by  military  personnel  from  the 
gaining  unit,  the  1 1th  Signal  Brigade. 

Most  of  the  usual  formal  environmental, 
roadability,  and  airlift  tests  were  deleted  in 
lieu  of  the  contractor  providing  an 
acceptable  analysis  confirming  compliance 
to  the  PM’s  satisfaction.  The  team  devoted 
a  tremendous  amount  of  rational  thought  and 
analysis  in  an  effort  to  eliminate  MILSPECs 


as  much  as  possible.  For  example, 
generators  are  normally  de-rated  to  allow 
operation  at  10,000  feet  above  sea  level. 

The  team  could  not  envision  a  scenario  in 
which  a  major  headquarters  would  be 
located  above  7,500  feet.  As  a  result,  the 
operating  specification  was  reduced  to  that 
altitude,  allowing  the  use  of  a  smaller 
commercial  generator  which  in  turn  saved 
several  hundred  pounds  in  a  severely 
constrained  weight  budget. 

Interoperability  testing  and  certification  was 
a  major  concern  as  these  terminals  would  be 
equipped  with  an  newly  developed,  digital 
Switch  Multiplexer  Unit  (SMU).  The  SMU 
had  to  be  fully  interoperable  with  the  Mobile 
Subscriber  Equipment  (MSE)  and  Tri- 
Service  Tactical  Communications  (TRI- 
TAC)  family  of  switches  as  well  as 
commercial  networks.  Just  30  days  into  the 
procurement,  OSD  published  new  guidance 
stating  that  no  new  switch  would  be  fielded 
unless  it  was  certified  by  the  Joint 
Interoperability  Test  Center.  Although  the 
PM  intended  to  thoroughly  test  this  newly 
developed  switch  and  to  have  it  certified  as 
interoperable  by  the  JITC,  the  team  knew 
that  the  schedule  did  not  permit  several 
months  of  dedicated  switch  testing  nor  did 
the  contractor  yet  have  an  operational  model 
for  parallel  testing. 

The  solution  for  interoperability  testing  was 
to  involve  the  JITC  early  and  up  front  and 
let  them  identify  their  interoperability 
concerns  for  incorporation  into  the  test  plan. 
High  level  management  in  JITC  agreed  to 
witness  interoperability  performance  at  each 
opportunity  throughout  the  test  program  and 
not  require  separate  formal  testing  prior  to 
fielding.  Designing  interoperability  in  and 
testing  it  as  part  of  first  article  testing  or 
during  other  types  of  tests  should  be  the 
model  for  interoperability  testing  in  the 
future. 
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The  elimination  of  some  of  the  formal 
environmental  testing  for  COTS/NDI 
procurements  is  an  acceptable  risk.  Many  of 
these  items  are  fully  capable  of  operating  in 
a  tactical  environment  if  properly  integrated 
into  a  suitable  platform.  In  the  case  of  the 
Tri-Band,  COTS/NDI  items  would  be 
mounted  in  a  MILSPEC  platform,  the 
HHMMWV,  inside  a  pallet  assembly  that 
provides  a  waterproof,  environmentally 
controlled  space.  Analysis  shows  the 
components  should  operate  satisfactorily. 
Experience  in  the  field  will  either  confirm  or 
deny  this  over  time. 

EXPEDITING  SOURCE  SELECTION 

Streamlining  did  not  stop  with  the  issuance 
of  the  RFP.  Since  the  acquisition  strategy 
demanded  that  the  contract  be  awarded 
under  conditions  of  full  and  open 
competition  using  a  best  value  source 
selection,  innovative  techniques  had  to  be 
implemented  to  reduce  the  normal  time 
required  for  proposal  evaluation, 
negotiation,  and  contract  award. 

The  Chairman  of  the  Source  Selection 
Evaluation  Board  (SSEB)  was  highly 
experienced  having  completed  five  major 
evaluations  over  the  previous  five  years.  He 
proposed  to  the  Source  Selection  Authority 
to  only  evaluate  key  discriminators  given  the 
limited  time  available.  Logistics  was  a 
factor  not  evaluated  during  the  source 
selection,  since  his  experience  indicated  that, 
although  important,  logistics  never  had  been 
a  tie-breaker  in  a  source  selection. 
Traditionally,  offerors’  proposals  state  their 
systems  are  fully  supportable  and 
consistently  underestimate  the  need  for 
repair  parts  in  order  to  reduce  their  bid. 
Instead,  offerors  were  given  a  ceiling  price 
for  the  initial  spares  and  maintenance 


support,  equalizing  the  logistics  cost  for  all 
proposals  and  were  required  to  submit  a 
Maintenance  Plan  for  the  PM’s  approval 
within  30  days  of  contract  award.  The 
program  management  capability  of  the 
offerors  also  was  not  evaluated  since  the 
known  potential  offerors  were  all  considered 
acceptable  based  upon  prior  history. 

The  offeror’s  proposal  was  limited  to  only 
100  pages  in  addressing  the  technical  factor 
in  order  to  reduce  superfluous  text,  thereby 
reducing  evaluation  time.  Many  of  the 
functional  matrix  experts  were  upset  at  this 
decision  because  they  felt  we  had  ignored 
their  particular  area  of  expertise.  One 
example  was  the  expert  in  Human  Factors, 
where  the  functional  support  element 
insisted  on  100  pages  for  their  area  alone. 
Other  functional  areas  that  lobbied  for  more 
data  during  source  selection  was  Logistics 
and  Electro-Magnetic  Interference  (EMI). 

The  time  consuming  task  of  accomplishing 
Performance  Risk  Assessment  during  source 
selection  had  to  be  expedited.  It  was 
decided  to  initiate  the  process  before  release 
of  the  RFP  by  inviting  all  interested  offerors 
to  submit  the  required  information  early. 

All  potential  offerors  were  requested  via  the 
EBB  on  December  28,  1993  to  submit  the 
required  information  by  January  10,  1994. 
This  preliminary  information  in  no  way 
limited  the  information  that  could  be 
submitted  in  the  formal  proposal.  However, 
the  early  “jump-start”  proved  to  be 
instrumental  in  meeting  the  schedule  as 
requested  information  from  Government 
sources  was  still  being  received  in  February. 

The  team  also  realized  that  there  was  not 
enough  time  to  ask  the  offerors  multiple 
rounds  of  questions  or  Items  for  Negotiation 
(IFNs)  during  source  selection.  As  a  result, 
IFNs  were  thoroughly  scrubbed  to  ensure 
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they  were  comprehensive,  clearly  written 
and  to  the  point.  Unfortunately,  offerors 
were  required  to  get  their  response  right  the 
first  time  or  risk  being  eliminated  from  the 
competition.  This  was  absolutely  necessary 
as  the  ambitious  schedule  did  not  allow  time 
for  a  second  round  of  IFNs  or  face  to  face 
discussions. 

The  source  selection  proceeded  ahead  of 
schedule  with  only  one  minor  amendment  to 
the  RFP,  issued  electronically  via  the  EBB, 
during  the  selection  period.  There  were  no 
official  requests  from  the  offerors  for 
extensions  of  time  in  order  to  meet  the 
source  selection  milestones.  This  was  due  to 
fact  that  the  vast  majority  of  the  offerors’ 
concerns  were  addressed  during 
development  of  the  RFP.  The  excellent 
communications  established  with  industry, 
facilitated  by  use  of  the  EBB,  made  this 
possible. 

The  high  quality  of  the  source  selection 
process  is  further  evidenced  by  the  fact  that 
despite  the  high  level  of  interest  by  industry 
resulting  in  ten  proposals,  the  contract  was 
awarded  ahead  of  schedule  with  no  protests. 
The  PM  attributes  this  to  continuous 
communications  with  industry  throughout 
the  process.  A  key  factor  in  mitigating  the 
risk  of  protests  is  taking  all  necessary 
measures  to  establish  good  communications 
with  industry.  The  Government  must  tell 
industry  how  their  proposals  will  be 
evaluated,  maintain  the  integrity  of  the 
source  selection  process,  and  provide  good 
debriefings  to  the  unsuccessful  offerors 
detailing  why  they  were  not  awarded  the 
contract. 

The  streamlined  source  selection  not  only 
reduced  Government  program  costs,  but  also 
reduced  industry’s  bid  and  proposal  costs. 
We  the  Government  frequently  fail  to 


appreciate  the  tremendous  costs  incurred  by 
industry  to  prepare  a  proposal  and  retain  the 
proposal  team  on  the  company  payroll  while 
the  Government  completes  the  source 
selection.  Several  of  the  unsuccessful 
offerors  commented  on  the  fact  that 
although  they  would  have  liked  to  have  won 
the  contract,  they  appreciated  the 
government  keeping  to  the  schedule  and 
completing  the  source  selection  in  record 
time,  thereby  minimizing  their  proposal 
costs. 

Reducing  industry’s  proposal  costs  may  also 
reduce  the  risk  of  a  protest  as  an  offeror  has 
made  a  smaller  investment  in  an  attempt  to 
win  a  contract.  One  offeror  commented  that 
he  had  a  proposal  team  on  the  payroll  for 
two  years  awaiting  award  of  another 
unrelated  contract.  He  went  on  to  state  that 
if  they  didn’t  win  the  contract,  he  had  no 
choice  but  to  protest  as  he  had  to  justify  his 
investment  to  his  management. 


THE  NEED  FOR  ADDITIONAL  TEAM 
MEMBERS 

Streamlining  acquisition  cannot  be 
accomplished  by  the  PM’s  team  alone.  The 
DA  and  DOD  staff  also  need  to  re-look  how 
they  execute  their  oversight  responsibility. 
Although  the  team  was  successful  in  pulling 
together  Team  Fort  Monmouth,  the  DA  Staff 
proved  to  be  reluctant  “to  take  to  the  field” 
and  join  the  team  in  this  procurement.  The 
DCSOPS  was  quick  to  call  in  October  1993 
to  tell  the  PM  to  begin  work.  However,  the 
formal  Directed  Procurement  letter  and 
funding  was  not  received  until  January  7, 
1994  five  days  before  RFP  release.  The 
PM’s  task  would  have  been  easier, 
especially  in  dealing  with  outside  agencies, 
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if  the  PM  had  written  direction  and  funding 
from  day  one. 

The  Assistant  Secretary  of  the  Army 
(Research,  Development,  and  Acquisition) 
staff  joined  “the  opposition  team”  late  in  the 
game.  In  late  June,  just  90  days  before 
delivery,  they  contacted  the  PM  and  insisted 
that  they  had  to  “get  control  of  the  program” 
as  the  PM  was  moving  too  fast.  Despite  the 
fact  that  this  was  a  DA  Directed 
Procurement  for  less  than  $10  million,  they 
insisted  that  a  Milestone  Decision  Brief  be 
provided  to  PEO  Command  and  Control 
Systems  (CCS)  who  had  just  been  made 
responsible  for  other  Commercial  Space 
Package  (CSP)  procurements.  Although  this 
decision  was  eventually  reversed,  it  cost  the 
team  two  man  weeks  of  effort  to  prepare  for 
and  present  briefings  on  why  a  formal 
Milestone  Decision  Briefing  to  PEO  CCS 
was  unnecessary. 

CURRENT  STATUS 

Although  the  streamlined  preparation  of  the 
PDP  and  award  of  the  contract  is  a  major 
success  story,  the  contractor  has  had  his 
problems  meeting  the  ambitious  delivery 
requirement  of  6  months  after  contract 
award.  Four  of  the  six  terminals  have  been 
delivered  to  Fort  Huachuca,  Arizona  where 
they  are  undergoing  a  period  of  testing  and 
fixing  by  the  contractor.  The  government 
has  demanded  and  received  consideration 
from  the  contractor  for  delays  and  continues 
to  demand  that  problems  be  fixed  before 
unit  training  can  begin.  Although 
streamlining  is  important,  the  PM  is 
adamant  that  he  will  not  field  a  system  to  the 
soldier  if  he  is  not  confident  it  will  operate 
properly. 

It  is  important  for  the  reader  to  understand 
that  problems  encountered  in  the  execution 


of  this  program  are  neither  attributable  to  the 
streamlining  innovations  nor  to  the  use  of 
COTS/NDI  components.  They  are 
attributable  to  the  contractor 
underestimating  the  challenges  of  integrating 
these  components  into  a  complex  battlefield 
system  within  the  extremely  short  period  of 
performance  required  by  the  contract. 

Also,  the  test  community  should  not  view 
the  current  problems  as  the  result  of  a  lack 
of  testing.  The  PM’s  abbreviated  testing 
program  successfully  detected  all  technical 
problems  prior  to  government  acceptance  of 
the  systems  and  has  successfully  identified 
the  deficiencies  for  correction  to  the 
contractor. 

CONCLUSION 

Adding  the  Tri-Band  terminal  to  the  U.S. 
Army's  communications  arsenal  will  permit 
the  use  of  widely  available  commercial 
satellites  as  required  by  recent 
Congressional  action  and  DOD  guidance. 
This  new  capability  will  reduce  the  demands 
on  the  highly  congested  military  satellite 
system.  In  addition  to  supporting  Army  and 
Joint  operational  requirements,  these 
terminals  will  also  be  an  important 
evaluation  tool  to  provide  valuable 
information  to  the  Materiel  Developer  for 
future  use  in  improving  procurements  of 
similar  items. 

The  rapid  acquisition  of  the  Prototype  Tri- 
Band  terminal  will  dramatically  increase  the 
Army's  war  fighting  capability  by  providing 
commercially  available  technology  to  the 
soldier  before  it  becomes  obsolete.  This 
procurement  also  represents  a  unique 
opportunity  to  demonstrate  the  acquisition 
community's  ability  for  rapid  prototyping 
and  streamlining  the  complex  procurement 
process.  PEO  Communications  Systems  is 
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meeting  the  challenges  in  today’s  new 
acquisition  environment  and  stands  ready  to 
rapidly  field  the  best  combat  systems 
possible  to  the  soldier  of  the  future. 


APPENDIX  1 


THE  SCHEDULE 

22  Oct  93-  PM  SATCOM  received  telephonic  notification  to  begin  planning  for  the  rapid 
acquisition  of  6  prototype  SHF  Tri-Band  Tactical  Satellite  Terminals 

25  Oct  93-PM  SATCOM  established  a  Special  Projects  Office  for  the  acquisition  using 
detailed  personnel 

5  Nov  93  -  Completed  first  draft  of  the  specification. 

8-10  Nov  93-PM  SATCOM  conducted  meetings  at  FT  Gordon,  GA  to  further  define  user 
requirements,  present  the  acquisition  strategy,  and  to  review  the  draft  specification. 

10  Nov  93  -  Placed  an  announcement  in  the  Commerce  Business  Daily  that  PM 
SATCOM  was  conducting  a  market  survey  for  the  Tri-Band  terminal  and  that  a  briefing  to 
industry  would  be  conducted  on  22  Nov  93. 

10  Nov  93  -  Placed  formal  solicitation  announcement  in  CBD. 

22  Nov  93  -  Conducted  briefing  to  industry.  Handed  out  the  second  draft  of  the 
specification  to  industry  which  incorporated  the  input  from  the  user  community  for  formal 
comment. 

23  Nov  93  -  Conducted  one-on-one  discussions  with  potential  bidders. 

7  Dec  93  -  Established  a  Tri-Band  account  on  the  existing  Electronic  Bulletin  Board 
(EBB)  used  by  PM  MILSTAR. 

9  Dec  93-Received  a  Memorandum  from  MG  Gray,  Commandant  of  the  Signal  Center 
detailing  the  minimum  essential  requirements  for  the  Tri-Band. 

10  Dec  93  -  Placed  second  draft  specification  and  the  Executive  Summary  on  EBB. 

15  Dec  93-  Established  a  Source  Selection  Board  which  included  a  select  core  of 
personnel  with  a  high  level  of  technical  expertise  and  extensive  experience  in  the  source 
selection  process. 

17  Dec  93  -  Placed  statement  of  work  (SOW)  and  the  third  iteration  of  specification 
which  incorporated  industry  comments  on  the  EBB 


/ 


243 


23  Dec  93  -Placed  draft  solicitation  sections  B,  L,  and  M  on  the  EBB.  The  addition  of 
these  sections  informed  industry  on  the  required  deliverables,  how  to  prepare  their  proposal  and 
how  the  proposal  would  be  evaluated. 

28  Dec  93-  The  KO  issued  an  EBB  message  requesting  all  interested  vendors  to  submit 
the  necessary  preliminary  information  for  the  government  to  evaluate  the  Performance  Risk 
Assessment  factor  in  advance  of  release  of  the  RFP  with  a  suspense  of  10  Jan  94. 

4  Jan  94-Conducted  a  Pre-Solicitation  Conference  and  one-on-one  discussions  for  all 
interested  vendors. 

6  Jan  94  -  Received  final  comments  from  industry  on  all  drafts  via  the  EBB. 

1 1  Jan  94  -  Completed  Procurement  Data  Package  (PDP). 

12  Jan  94  -  Issued  Request  For  Proposal  (RFP). 

1 1  Feb  94  -  Received  proposals  from  Industry.  However,  deadline  extended  to  14  Feb  94 
due  to  a  major  east  coast  snow  storm. 

25  Feb  94  -  Completed  initial  proposal  evaluation. 

1  Mar  94  -  Conducted  initial  competitive  range  brief  to  Source  Selection  Authority 

(SSA). 

1  Mar  94  -  Released  Items  for  Negotiation  (IFN). 

8  Mar  94  -  Received  IFN  responses. 

8  Mar  94  -  Began  Interim  Evaluation. 

15  Mar  94  -  Completed  Interim  Evaluation. 

17  Mar  94  -  Briefed  SSA  on  interim  evaluation  findings. 

17  Mar  94  -  Requested  Best  and  Final  Offer  (BAFO).  Also  the  KO  sent  out  model 
contract  to  offerors  still  in  competition  and  required  they  sign  the  contract  and  return  with  the 
BAFO. 

21  Mar  94  -  Received  BAFOs. 

22  Mar  94  -  Completed  final  evaluation. 

23  Mar  94  -  Final  briefed  to  SSA. 

24  Mar  94  -  Awarded  contract  7  days  ahead  of  our  most  optimistic  schedule 
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APPENDIX  2 


CONTRACT  SYNOPSIS 


A  Firm  Fixed  Price  contract  for  six  Prototype  Tri-Band  SHF  Tactical  Satellite  Terminals  was 
awarded  to  GTE  on  24  March  1994  in  the  amount  of  $7,756,801.  The  basic  contract  includes  the 
hardware,  initial  spares,  training,  a  systems  level  operator's  manual  and  a  time  and  materials 
effort  for  contractor  maintenance  and  support  for  the  first  year  (FY95).  Delivery  of  all  six 
terminals  is  required  not  later  than  30  September  1994.  In  addition  to  the  basic  contract,  ceiling 
prices  were  established  for  four  1-year  depot  level  maintenance  options  for  FY96  through  FY99. 
These  options  are  for  time  and  materials  only  and  are  to  be  the  responsibility  of  the  gaining 
organization.  The  ceiling  prices  established  with  adjustments  for  inflation  are  as  follows: 


FY  96  $313,700 
FY  97  $320,600 
FY  98  $327,700 
FY  99  $334,900 


The  contractor  is  required  to  provide  depot  level  maintenance  for  the  non-GFE  components  and 
technical  support  in  the  field  throughout  the  life  of  the  terminals  (5  years). 
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A  Prince  in  Commercial  Rags 


Robert  E.  Lloyd,  CPCM 
U.S.  Department  of  State 


ABSTRACT 

One  of  the  most  widely  discussed 
initiatives  in  recent  years  is  the  proposed 
use  of  commercial- style  contracting 
procedures  in  the  Federal  Government. 
Under  this  approach,  the  State  would  act 
in  the  same  manner  as  a  private  buyer, 
acquiring  needed  supplies  and  services 
more  efficiently,  as  if  it  were  not  a 
sovereign.  This  paper  analyzes  the 
foundations  of  the  concept  in  terms  of 
legal  precedent,  political  economy,  and 
game  theory  in  an  effort  to  determine  its 
likelihood  of  success. 

The  Federal  Government  serves  two 
roles.  It  acts  as  a  contracting  party  as 
any  other  enterprise  in  market  exchange, 
but  it  also  bears  the  privileges  of 
sovereignty.  The  paper  seeks  to  portray 
the  divergent  views  of  the  State  as 
sovereign  and/or  contractor  and  thereby 
help  explain  the  apparent  lack  of 
progress  in  implementing  commercial 
contracting  methods.  The  dual  nature  of 
sovereignty  in  the  United  States 
necessarily  complicates  any  proposals  for 
reform  of  the  current  acquisition  system 
based  on  best  practices  of  industry. 

INTRODUCTION 

The  Vice  President's  National 
Performance  Review  has  recommended 
that  Government  contracting  regulations 
"foster  commercial  practices,"  as  part  of 


an  effort  to  make  the  Federal 
Government  work  better  and  cost  less.1 
Once  again,2  Federal  agencies  are  being 
asked  to  look  to  the  private  sector  for 
good  ideas  to  adopt  in  Government. 
Rather  than  addressing  which  buying 
procedures  the  Government  should  adopt 
or  how  it  should  acquire  commercial 
items,  this  paper  analyzes,  at  a  more 
fundamental  level,  how  the  State  should 
be  regarded  in  a  contractual  setting.  The 
Federal  Government's  twin  role  as  both 
sovereign  and  contractor  must  be 
explored  as  a  first  step  before  starting 
any  conclusive  discussion  of  the  ability  or 
desirability  of  the  State  to  emulate 
commercial  contracting  practices. 

THE  STATE  AS  CONTRACTOR 

If  the  State  were  omnipotent,  there 
would  be  no  need  for  Government 
contracts.  The  Federal  Government 
could  simply  obtain  what  it  needed  by 
seizure,  condemnation,  or  other  coercive 
measures.3  The  United  States,  however, 
has  a  system  of  limited  government,  so 
when  the  Federal  Government  needs 
something  from  an  outside  source  to 
discharge  its  assigned  functions,  it  must 
necessarily  contract  for  those  needs  and 
enter  the  commercial  market. 

Recently,  the  Government  even  issued  a 
statement  as  to  the  limits  of  its  own 
authority  to  contract.  Office  of  Federal 
Procurement  Policy  Letter  92-1  outlined 
areas  where  the  Government  may  not 
contract  out,  i.e.,  inherently 
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governmental  functions.  If  Federal 
agencies  were  truly  sovereign,  there 
would  be  no  such  limitation. 

The  courts  have  held  that  the 
Government,  by  entering  into  a  contract, 
has  rights  and  responsibilities  similar  to 
those  of  a  private  person  who  is  a  party 
to  a  contract.4  As  a  contractor. 
Government  must  act  fairly.  The 
Government  has  been  prevented  from 
taking  an  action  in  its  sovereign  capacity 
where  the  action  would  violate 
elementary  rules  of  fairness.5  The 
Claims  Court  has  ruled  that  our 
government  is  subject  to  general 
common  law  contracting  rules,  which 
limit  the  State's  power,  because  entry 
into  the  field  of  contracts  is  not  like  the 
Government's  creation  of  rights  and 
entitlements  in  other  fields.6  This  is 
consistent  with  Hayek's  view  that  the 
service  functions  of  government  (such  as 
contracting)  should  be  separated  from 
the  law  enforcement  and  national  defense 
realm.  The  authority  of  the  latter  should 
not  be  conferred  upon  the  former,  and 
the  State  should  have  no  special 
privileges  in  administering  its  programs.7 

The  State  cannot  adopt  as  its  policy  the 
repudiation  of  debts  or  the  destruction  of 
contracts,  or  for  that  matter  deny  the 
means  to  enforce  them.8  Although 
writing  in  a  dissenting  opinion,  Judge 
Jones  best  expressed  this  overall  position 
on  the  limits  of  Government  action  in  a 
contractual  setting:  "Just  because  the 
Government  has  the  sheer  power  to 
claim  that  a  certain  action  was  taken  in 
its  asserted  sovereign  capacity  does  not 
mean  that  a  Government  agency  may 
disregard  its  voluntary  contract 
obligations,  nor  that  it  may  ride 
roughshod  over  the  citizens'  rights  by 


simply  donning  the  cloak  of  immunity 
without  showing  a  necessity  for  doing 
so."9 

The  voluntariness  of  contracting  applies 
to  both  the  State  and  its  counterpart 
contractor,  since  both  reach  agreement 
voluntarily  in  our  capitalist  economic 
system.  Further,  it  has  been  said  that  the 
moral  legitimacy  of  the  State  depends  on 
its  fulfillment  of  its  contractual 
obligations.10  This  principle  is  evident  in 
the  longstanding  law  of  breach  of 
Government  contracts,  for  recovery  is 
based  on  the  principle  of  restitution 
where  that  remedy  would  be  available  if 
the  Government  were  a  private 
contracting  party.11  Contractors  are 
likewise  not  required  to  comply  when  the 
Government  makes  cardinal  changes  to  a 
contract.12  Instead  of  being  subject  to 
the  whims  of  an  omnipotent  buyer,  a 
contractor  selling  to  the  Government 
may  recover  if  the  Contracting  Officer 
acted  in  an  arbitrary,  capricious,  or 
malicious  manner.13 

The  Supreme  Court  has  distinguished 
between  the  Federal  Government  acting 
as  a  market  participant  versus  a  market 
regulator.14  The  standards  of  behavior 
are  not  the  same  in  both  realms.  In  a 
related  development,  boards  of  contract 
appeals  have  recently  begun  to  recognize 
that  a  Government  agency,  though 
presumed  to  act  in  good  faith,  is  an 
organization  motivated  to  behave  in  the 
same  way  as  any  other  private  party 
seeking  a  contractual  advantage.15 

The  law  governing  most  commercial 
contracts,  the  Uniform  Commercial  Code 
(UCC),  even  applies  on  occasion  to 
Federal  contracts.  It  is  well  established 
that,  in  the  absence  of  a  statute  or 
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regulation,  the  interpretation  and 
construction  of  Government  contracts 
will  be  governed  by  principles  of  general 
contract  law.  The  UCC  is  regarded  by 
the  courts  and  boards  of  contract  appeals 
as  law  applicable  to  Government 
contracts.16  Although  Federal  laws  and 
regulations  generally  cover  most 
contracting  situations,  the  fact  that  the 
UCC  does  apply  in  some  circumstances 
is  another  example  of  how  the  State  is 
sometimes  treated  as  just  another 
contractor. 

An  additional  way  in  which  the  State  is 
regarded  in  the  same  manner  as  a  private 
concern  is,  oddly  enough,  in  the 
application  of  the  so-called  sovereign 
acts  doctrine.  The  Supreme  Court  has 
decided  that,  although  the  doctrine  of 
inherent  sovereignty  does  not  apply  to 
the  Federal  Government,  the  United 
States,  as  a  party  to  a  contract,  is  not 
liable  for  breach  of  its  "proprietary  and 
corporate  contracts  due  to  its  public  and 
general  acts  as  a  sovereign."17  In  other 
words,  if  Agency  X  has  a  contract  with 
Company  Y,  and  Agency  Z  (or  even  an 
office  other  than  the  contracting  office  in 
Agency  X,  or  Congress)  takes  an  action 
of  a  public  and  general  nature  (i.e.,  not 
specific  to  the  contract  at  hand)  that 
affects  Company  Y's  contract  (typically, 
increasing  costs  or  interfering  with  the 
work),  then  the  sovereign  acts  theory 
bars  recovery  by  Company  Y  for  this 
disruption  to  its  contract. 

One  reason  argued  by  the  courts  in  favor 
of  this  concept  is  that  the  Government 
should  not  be  liable  contractually  for  acts 
taken  in  a  sovereign  capacity  for  the 
public  good.18  The  Claims  Court  has 
ruled  that  the  sovereign  cannot  forever 
waive  its  rights  to  exercise  its  sovereign 


powers  unless  it  expressly  reserves  the 
right  to  do  so  in  the  contract.19  It  is  said 
that  to  rule  otherwise  would  mean  that 
one  Congress  could  lock  the  Government 
into  binding  contracts  involving  areas 
that  a  future  Congress  would  be 
powerless  to  legislate  on,  producing  a 
result  in  which  the  legislature  could  not 
be  responsive  to  its  citizens.20 
Additionally,  it  has  been  claimed  that  the 
sovereign  acts  defense  is  necessary  so 
that  private  parties  will  not  be  able  to 
obtain  immunity  from  State  regulation 
simply  by  making  private  contractual 
agreements.21  Nevertheless,  the  doctrine 
has  its  limits.  Contracts,  even  those  with 
the  Federal  Government,  are  not  subject 
to  unlimited  modification  under  the 
police  powers  of  the  State.  The  State 
may  only  impair  contracts  if  reasonably 
necessary  to  serve  an  important  public 
purpose.22 

Another  rationalization  for  this  treatment 
of  contractors  is  that  in  the  case  of 
private  parties  to  a  contract,  the  burden 
of  a  change  in  the  law  is  usually  borne  by 
the  party  on  which  it  falls,  absent  specific 
contract  wording  to  the  contrary.  Under 
the  sovereign  acts  doctrine,  parties 
contracting  with  the  Government  will 
only  have  rights  against  the  State  that 
would  be  available  against  a  private 
company.  If  a  contractor  brought  an 
action  against  a  private  corporation  due 
to  acts  of  the  Federal  Government,  it 
could  not  be  sustained.  The  United 
States  is  not  held  to  a  greater  liability 
than  other  contractors  in  court.23 

Although  the  sovereign  acts  doctrine 
superficially  (and  conveniently,  for  the 
Government)  considers  the  State  as 
simply  another  corporate  entity,  the  fact 
remains  that  from  a  contractor's 
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perspective,  the  sovereign  acts  doctrine 
may  also  be  viewed  as  yet  another 
weapon  in  the  State's  arsenal  of  defenses. 
Despite  the  fact  that  the  Federal 
Government  is  not  monolithic,24  and  one 
agency  is  not  responsible  for  the  acts  of  a 
sister  agency,  to  a  contractor  the 
sovereign  acts  doctrine  absolves  the 
State  for  liability  from  its  actions  as  a 
whole.  The  theory  offers  little 
consolation  to  affected  firms,  even 
though  its  purported  justification  is  to 
treat  Government  and  contractor  as 
equal  trading  partners  in  the  market. 

A  further  defect  in  the  sovereign  acts 
doctrine  is  that  it  encourages 
Congressional  profligacy.  If  a  legislature 
is  empowered  to  pass  laws  without 
regard  to  their  impact  on  existing 
contracts,  there  will  be  little  stopping  the 
enactment  of  legislation  that  will  be 
burdensome  and  costly  to  private 
industry.  This  can  hardly  be  consistent 
with  ongoing  attempts  to  obtain  more 
competition  in  Government  contracts. 
Commentators  have  also  pointed  out  that 
there  is  no  limit  to  the  types  of  acts  that 
might  potentially  qualify  for  protection 
under  the  sovereign  acts  doctrine, 
ranging  from  protection  of  the 
environment  to  controlling  inflation.25 
The  standard  of  "public  and  general  act" 
has  been  roundly  criticized  as  a  weak 
protection  for  contractors,  since  all 
legislation  is  supposed  to  be  for  "the 
public  good."26 

The  sovereign  acts  defense  rests  on  the 
theory  that  Government  actions  may  be 
either  of  two  characters:  acts  in  a 
proprietary  capacity  (as  a  contractor),  or 
acts  in  a  sovereign  capacity  (as  the 
State).27  Nevertheless,  in  one  sovereign 
acts  case,  the  Court  of  Claims  insisted 


that  "the  two  characters  which  the 
Government  possesses  as  a  contractor 
and  as  a  sovereign  cannot  be  thus 
fused."28  Theory  aside,  the  sovereign  acts 
doctrine  is  an  example  of  how  the  dual 
public/private  nature  of  the  State 
produces  unpleasant  results  from  an 
industry  perspective  in  the  field  of 
Government  contracting. 

Another  instance  of  how  dealing  with  the 
State  as  a  contractor  results  in  a 
disadvantage  to  private  contractors 
concerns  the  Government's  right  to  set 
terms  and  conditions  in  its  contracts. 
The  landmark  Perkins  v.  Lukens  Steel 
Company  case  that  denied  standing  to  a 
contractor  suing  the  Government  over 
objectionable  solicitation  provisions 
presented  the  Supreme  Court's  position 
that  the  Federal  Government  "enjoys  the 
unrestricted  power  to  produce  its  own 
supplies,  to  determine  those  with  whom 
it  will  deal,  and  to  fix  the  terms  and 
conditions  upon  which  it  will  make 
needed  purchases."29  What  is  curious  is 
the  reasoning  given  by  the  court  for  this 
stance,  namely,  that  the  same  outcome 
would  occur  among  private  individuals, 
who  are  free  to  write  contracts  in  any 
manner  they  please.  Although  this  is 
largely  true  under  the  UCC,30  it  presents 
a  paradox,  for  if  we  treat  the  State  as  a 
contractor,  the  Government  arguably 
gains  unlimited  power  in  contracting. 
Those  who  would  speak  in  favor  of 
adopting  commercial  practices  in 
Government  contracting  should  be 
mindful  of  this  result. 

THE  STATE  AS  SOVEREIGN 

Despite  the  legal  precedent  for  holding 
the  State  to  the  same  standards  of 
conduct  as  a  private  firm,  there  is  a 
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corresponding  body  of  thought  that  holds 
a  contrary  view.  In  the  early  years  of  the 
Republic,  Alexander  Hamilton  wrote: 
"Contracts  between  a  nation  and 
individuals  are  only  binding  on  the 
conscience  of  the  sovereign,  and  have  no 
pretension  to  a  compulsive  force.  They 
confer  no  right  of  action  independent  of 
the  sovereign  will."31  The  reason  for  this 
argument  was  that  even  before  the 
Constitution  was  passed,  all  States  had 
enjoyed  sovereign  immunity.32  This 
concept  has  survived,  but  only  to  a 
limited  degree  in  Government  contracts 
of  this  century.  A  famous  instance  in 
which  the  Federal  Government  was  ruled 
not  to  be  just  another  contracting  party 
was  the  Federal  Crop  Insurance 
Corporation  case  of  the  1940s.  In  this 
case,  the  Supreme  Court  denied  standing 
to  a  party  bringing  suit  against  a 
Government-sponsored  enterprise, 
explaining  that  the  Government  is  not 
just  another  private  litigant  and  cannot  be 
charged  with  liability  as  the  sovereign.33 
Despite  the  clarity  of  its 
pronouncements,  the  case  is  of  little 
import  for  contemporary  Federal 
acquisition,  because  in  the  past  few 
years,  the  Federal  Government  has 
increasingly  waived  its  sovereign 
immunity  in  contracting. 

In  addition  to  providing  special  contract 
dispute  resolution  bodies,  Congress  has 
passed  laws  greatly  expanding  the  bid 
protest  mechanism,  ostensibly  for  the 
purpose  of  making  the  Federal 
acquisition  system  more  fair  and 
responsive  to  the  complaints  of 
commercial  firms  who  are  unsuccessful 
in  winning  contracts.34  But  does  the 
Government's  stepping  down  from  a 
position  of  sovereign  power  make  it 
more  like  a  commercial  buyer?  The 


answer  is  clearly  no,  because  commercial 
sellers  have  no  rights  of  protest  against 
contract  awards  made  by  private  buyers, 
whether  under  the  Uniform  Commercial 
Code  or  common  law  of  contract. 
Ironically,  by  making  the  State  less 
"sovereign"  in  power,  we  have  also  made 
it  less  resemble  a  commercial  contractor. 

Several  authors  have  compared 
commercial  and  Government  contracts 
and  found  key  areas  of  difference.  The 
most  frequently  cited  "burdens"  are  the 
sovereign's  right  to  terminate  for 
convenience,  and  its  power  to  make 
unilateral  changes  to  contracts.35  A 
review  of  commercial  contracts  shows 
this  position  to  be  weak.  Commercial 
contract  terms  and  conditions  used  by 
several  firms  include  changes  and 
termination  for  convenience  clauses 
nearly  identical  to  those  included  in  the 
FAR.36  This  may  be  partly  due  to  the 
fact  that  the  UCC  allows  the  parties  to 
create  their  own  contracts  within  very 
wide  boundaries.  Adoption  of  UCC 
principles  of  commercial  contracting 
would  produce  surprisingly  unpleasant 
results  for  private  industry  in  other  areas 
as  well.  Under  the  UCC,  there  is  also  no 
need  for  public  notice  of  upcoming 
contracts,  competition,  or  even 
necessarily  a  written  solicitation.37 
Obviously,  in  this  respect,  to  regard  a 
Government  agency  as  a  sovereign  with 
unique  contracting  rules  is  to  the 
advantage  of  private  firms  over  the 
alternative  of  following  commercial 
practices. 

One  of  the  true  manifestations  of 
sovereignty  in  Federal  contracting 
involves  the  Government  contractor 
defense.  In  many  cases,  firms  working 
on  Federal  contracts  have  been  held  to  be 
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immune  from  tort  liability  if  an  injury  or 
death  occurs  as  a  result  of  a  failure  of 
equipment  produced  under  a 
Government  contract.38  The  justification 
for  this  defense  is  that  a  private  firm 
should  be  protected  if  it  is  simply 
executing  the  will  of  the  State.  The 
underlying  assumption  is  that  without 
immunity,  no  one  would  be  willing  to 
compete  for  Government  contracts  of 
such  a  nature.  Nevertheless,  despite  the 
apparent  fairness  of  this  doctrine  from 
the  perspective  of  the  private  contractor, 
it  has  been  criticized  for  victimizing 
innocent  civilians  for  occurrences  that 
could  either  be  covered  by  insurance  or 
should  be  factored  into  a  firm's  cost- 
benefit  analysis  when  deciding  to  enter 
the  Government  contracting  arena.39 
Such  criticism  illustrates  again  how  our 
views  of  the  sovereign  are  clouded  by 
ambiguity. 

The  paradox  of  the  State  as  contracting 
party  appears  in  many  forms,  including  a 
Court  of  Claims  decision  regarding  the 
role  of  the  Government's  Contracting 
Officer.  The  court  opined  that  the 
Contracting  Officer  serves  a  dual  role, 
that  of  a  representative  of  the  State  and 
that  of  an  impartial  arbitrator  of  disputes. 
He  or  she  "must  not  act  as  a 
representative  of  one  of  the  contracting 
parties,  but  as  an  impartial,  unbiased 
judge."  Nevertheless,  the  Contracting 
Officer's  findings  of  fact  are  presumed  to 
be  conclusive,  absent  evidence  that  they 
are  arbitrary  or  capricious.40  Even  at  the 
most  individual  level,  legal  precedent 
holds  the  State  to  a  different  standard 
than  ordinary  commercial  parties. 

Along  these  lines,  one  of  the  most 
frequently  heard  excuses  for  having 
different  rules  for  contracting  with  the 


sovereign  than  for  commercial 
contracting  is  that  Federal  contracts 
spend  taxpayer  money.  The  argument 
fails  upon  further  analysis.  The 
assumption  put  forth  is  that  the  State 
must  be  especially  careful  in  how  it 
spends  its  tax  dollars.  The  implication  is 
that  when  an  individual  citizen  spends  his 
or  her  own  money,  there  is  no  concern 
for  how  wisely  it  is  spent.  Few  people 
would  agree  with  this  proposition. 
Alternatively,  this  theory  implies  that 
commercial  buyers  are  under  no  pressure 
to  spend  shareholder  dollars  wisely. 
Clearly,  this  is  not  the  case.  Therefore, 
the  proposition  that  Government  should 
be  accorded  deferential  treatment  simply 
because  it  is  in  charge  of  protecting 
taxpayer  funds  is  a  specious  one. 

There  remain  valid  reasons  why  the  State 
does  not  and  cannot,  under  current  law, 
behave  more  like  a  private  firm  when  it 
contracts.  In  the  purely  commercial 
world,  there  are  several  governance 
structures  available  to  help  preserve  the 
existence  of  trade  relationships.  These 
include  trading  only  with  one's  friends, 
concerns  about  loss  of  future  sales,  or 
use  of  vertical  integration.  For  the 
Federal  Government,  only  one 
governance  structure  is  available:  formal 
contracts,  upon  which  almost  complete 
reliance  must  be  placed.  It  is  only  natural 
for  the  State  to  resort  to  lengthy 
contracts  that  bear  little  resemblance  to 
private  agreements.41 

Yet  at  a  more  basic  level,  the  paradox  or 
contradiction  inherent  in  U.S. 
Government  contracting  merely  echoes 
the  foundation  of  our  form  of 
government  itself:  the  principle  of 
popular  sovereignty.  In  America, 
fairness  and  certain  standards  of  behavior 
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are  expected  from  the  State  because  ours 
is  a  government  founded  on  the  premise 
that  all  power  resides  in  the  people, 
except  where  government,  as  a  creature 
of  its  human  founders,  has  been  granted 
limited  authority  in  the  service  of  the 
populace.42  As  the  doctrine  of  divine 
right  of  Kings  came  into  disrepute,  it 
gave  way  to  the  idea  of  the  sovereignty 
of  the  people  in  seventeenth  century 
England,  and  was  popularized  during  the 
American  Revolution.  Edmund 

Morgan43  points  out  that  both  theories 
require  the  suspension  of  disbelief  and 
serve  as  convenient  fictions  to  persuade 
the  many  to  submit  to  the  government  of 
the  few. 

There  is  a  circular  logic  in  creating  a 
government  comprised  of  citizens 
representing  "the  people"  and  thereby 
posing  the  question  of  who  is  left  to 
protect  the  citizens  from  the  arbitrary 
acts  of  a  government  that  is  popularly 
elected  or  supposedly  controlled  by 
elected  officials.  Another  author  refers  to 
the  "dilemma  of  representative 
government"  and  describes  its  basic 
precept  as  self-contradiction,  hypocrisy, 
and  illusion,44  for  to  complain  about 
government  in  a  nation  founded  on  self- 
government  is  to  criticize  oneself.  This 
tension  lies  at  the  core  of  democratic 
government.45 

The  success  of  popular  sovereignty  has 
always  rested  on  a  fragile  base  of 
representation,  with  power  exercised 
only  by  agents  of  the  populace  who  were 
simultaneously  representatives  of  the 
people  and  wielders  of  supreme  power.46 
This  same  ambiguity,  derived  from  the 
contractual  basis  of  American 
democracy,  carries  over  to  Government 
contracting  as  well.  The  State  is  expected 


to  be  a  contracting  party  and  a  sovereign 
at  the  same  time.  Those  who  would 
complain  about  the  State's  inefficiency 
compared  to  commercial  contracting 
would  also  demand  the  Federal 
Government  to  be  fair  and  consistent  in 
its  practices,  even  though  these  two 
goals  are  often  in  conflict.47 

Economic  philosophers  have  pointed  out 
this  conflict  in  several  aspects.  Bertrand 
de  Jouvenel  notes  that  although  the 
individual  may  be  sovereign  in  a 
democracy,  his  sovereignty  is  a  power 
almost  always  held  in  abeyance.48 
Holcombe  states  that  the  danger  in  social 
contract  theory  inherent  in  representative 
democracy  is  that  it  can  be  used  to  justify 
coercion  by  arguing  that  those  being 
coerced  agreed  to  a  social  contract  and 
now  must  abide  by  the  Government's 
rules.49  This  dilemma  is  just  as  acute  for 
the  commercial  firm  that  pursues 
Government  business. 

WHY  PLAY  THE  GAME? 

After  looking  at  the  ways  in  which  the 
State  may  invoke  its  sovereign  powers, 
we  may  find  it  only  natural  that 
companies  should  refuse  to  compete  for 
Government  contracts.  Indeed, 

regulatory  controls  have  been  cited  as  a 
barrier  to  entry  in  Federal  acquisition.50 
But  the  reality  is  that  the  contracting 
agency's  status  as  a  sovereign  does  not 
figure  prominently  when  companies  are 
surveyed.  Lamm's  study  of  over  400 
firms  revealed  that  the  most  frequently 
cited  reasons  why  firms  refuse  DoD 
business  were,  by  far,  burdensome 
paperwork  and  Government  bidding 
methods,51  and  not  a  perceived  inequality 
of  bargaining  power. 
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If  contracting  with  the  sovereign  is  such 
a  bad  deal,  then  why  are  contractors  still 
interested  in  Government  business? 
Epstein  argues  that  it  is  improper  for  the 
Federal  Government  to  impose 
conditions  on  firms  or  private  parties  in  a 
contract  that  would  elsewhere  be 
considered  unconstitutional.52  Although 
his  argument  is  compelling,  few 
companies  seem  to  be  listening.  Perhaps 
this  is  because,  as  Epstein  confesses, 
each  firm  acting  alone  may  feel  that  it  is 
in  its  interest  to  give  up  some 
constitutional  right,  even  though  as  a 
group,  all  contractors  would  reach  the 
opposite  conclusion.  Hence  firms 
continue  to  do  business  with  the 
Government  despite  objectionable  terms 
and  conditions.  Another  explanation  for 
this  behavior  lies  in  game  theory,  which 
can  be  employed  to  show  how  it  is 
rational  for  contractors  to  submit  to  the 
Government's  terms  and  conditions  in 
contracting. 

A  Prisoners  Dilemma  scenario  illustrates 
the  choices  facing  contractors  in  deciding 
whether  to  compete  for  Government 
contracts.  In  the  following  diagram,  to 
comply  means  to  compete  for  the 
Government  contract  on  the 
Government's  terms,  as  stated  in  the 
solicitation.  To  resist  means  to  refuse  to 
abide  by  these  terms  and  not  to  submit  an 
offer  at  all.53  The  diagram  is  a  two- 
person,  zero-sum  game  in  which  there 
are  only  two  possible  competitors  for  a 
Government  contract  (a  situation  which 
may  not  be  the  most  common  but  one 
which  is  certainly  not  unknown). 


A 

Comply 

Resist 

B 

Comply 

7.5,  7.5 

II 

5,  10 

I 

Resist 

10,  5 

III 

5,5 

IV 

An  explanation  of  the  values  in  each 
quadrant  or  payoff  box  is  as  follows: 

I:  This  quadrant  represents  the  fact 

that  if  A  refuses  to  deal  on  the 
Government's  terms,  B  will  become  the 
sole  source  supplier  to  the  Government, 
and  their  revenues  become  5  and  10 
monetary  units,  respectively.  Even  if  this 
condition  did  not  hold  and  there  were 
more  than  two  firms  in  the  relevant 
market,  B  would  still  have  a  better 
chance  at  getting  a  Government  contract, 
in  light  of  the  reduced  competition 
resulting  from  A's  departure. 

II:  If  both  firms  go  along  with  the 

Government's  terms  and  conditions, 
however  unreasonable,  they  have  a 
chance  to  increase  their  revenue  (to  7.5 
units)  through  participation  in  the  market 
for  Government  contracts.  Note  that  the 
potential  payoff  is  less  than  the  highest 
value  available  in  quadrants  I  and  III, 
because  both  A  and  B  must  compete  for 
the  contract,  and  each  presumably  has 
only  a  50%  chance  of  winning.54 

Ill:  This  is  converse  result  to 

quadrant  I,  where  B  opts  out,  leaving  A 
in  line  for  the  contract. 


IV:  In  this  quadrant  (as  well  as  in 

sectors  I  and  III),  the  value  5  represents 
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the  fact  that  each  firm  has  commercial 
business  and  is  not  completely  dependent 
upon  Government  contracts,  although  it 
is  recognized  that  some  firms  do  not  fit 
this  profile.  If  both  firms  opt  out  of  the 
market  for  Government  contracts,  they 
will  still  have  income.  Perfect 
competition  need  not  exist  in  the 
commercial  market  for  this  result  to  hold 
true. 

In  addition  to  the  many  instances  where 
Government  buying  rules  are  more 
favorable  to  companies  than  commercial 
practices  under  the  UCC,  game  theory 
shows  us  why  it  is  understandable  for 
firms  to  seek  Government  business  even 
when  they  do  not  like  the  terms.  While 
some  may  claim  that  commercial 
practices  are  better  or  more 
advantageous  to  private  industry, 
companies  still  find  it  rational  to  enter  the 
field  of  Government  contracting. 

CONCLUSION 

The  premise  of  contemporary  efforts  to 
make  Government  more  business-like  in 
its  contracts  is  that  the  State  and  the 
private  corporation  can  be  considered 
equals.  Legal  precedent  supports  the 
view  of  the  State  as  contractor  in  several 
respects,  but  there  are  still  remnants  of 
sovereign  immunity  and  other  areas 
where  the  Federal  Government  as  buyer 
cannot  be  placed  on  the  same  footing 
with  its  commercial  partner.  This 
inconsistency  is  not  surprising,  given  the 
nature  of  our  government  where  power 
and  authority  are  established  on  the 
paradoxical  basis  of  popular  sovereignty. 
It  then  becomes  understandable  for 
private  industry  to  complain  of  Federal 
acquisition  rules  while  at  the  same  time 
finding  it  rational  to  play  the  game. 


To  borrow  a  phrase,  the  push  for 
commercial-style  contracting  by  the 
Federal  Government  is  a  policy  at  war 
with  itself.55  The  adoption  of 
commercial  contracting  procedures  might 
streamline  Federal  acquisition  at  the 
expense  of  private  industry.  Acquisition 
policy  makers  should  realize  that  a 
tradeoff  is  involved  in  discarding 
Government-unique  procedures  for 
industry  practice.  Whether  we  want  the 
prince  to  wear  commercial  rags  rather 
than  the  sartorial  trappings  of 
sovereignty  is  a  question  demanding  that 
we  look  beyond  appearances.  Before  we 
blindly  accept  the  proposal  that 
commercial  practices  will  solve  the 
problems  of  today's  acquisition  system, 
we  should  be  aware  of  the  larger 
implications  that  face  us  when  we  view 
the  State  as  a  one-dimensional  entity.  As 
long  as  the  State  serves  a  dual  role,  the 
application  of  commercial  practices  to 
Federal  contracts  will  remain  an  elusive 
goal. 

ENDNOTES 

[The  views  expressed  are  solely  the 
author's  and  do  not  necessarily  represent 
those  of  any  U.S.  Government  agency. 
The  title  of  the  paper  is  derived  from  a 
passage  in  James  O.  Mahoy,  ed., 
Government  Contract  Law 

(USAF/AFIT,  1979),  27  ] 

1  A1  Gore,  From  Red  Tape  to  Results: 
Creating  a  Government  that  Works 
Better  and  Costs  Less  (Report  of  the 
National  Performance  Review: 

September  7,  1993,  GPO),  28. 

2  The  applicability  of  private  industry 
techniques  has  long  been  questioned  in  a 


257 


governmental  context;  see,  e.g.,  Brian  W. 
Rapp,  "You  Can't  Manage  City  Hall  the 
Way  You  Manage  General  Motors,"  in 
Frederick  S.  Lane,  ed.,  Current  Issues  in 
Public  Administration  (New  York:  St. 
Martin's,  1978),  416-421. 

3  Mahoy,  27.  Some  would  argue  that 
even  a  despotic  government  would  agree 
to  limit  itself  in  the  interest  of  self- 
preservation,  to  lull  the  populace  into  a 
false  sense  of  security;  Anthony  de  Jasay, 
The  State  (New  York:  Basil  Blackwell, 
1985),  191-192. 

4  G.L.  Christian  and  Associates  v.  U.S.. 
312  F.2d  418,  423  (1963). 

5  United  States  v.  Lazy  FC  Ranch.  48 1 
F.2d  985,  989  (1973).  See  also  FAR 
50.302-l(b). 

6  Torncello  v.  U.S..  681  F.2d  756,  762- 
763  (1982). 

7  Friedrich  A.  Hayek,  Law.  Legislation, 
and  Liberty.  Volume  3:  The  Political 
Order  of  a  Free  People  (University  of 
Chicago  Press,  1979),  41-47. 

8  United  States  Trust  Company  of  New 
York  v.  New  Jersey.  431  U.S.  1,  22 
(1976). 

9  Air  Terminal  Services.  Inc,  v.  U.S.. 

165  Ct.  Cl.  525,  536  (1964). 

David  B.  Toscano,  "Forbearance 
Agreements:  Invalid  Contracts  for  the 
Surrender  of  Sovereignty,"  Columbia 
Law  Review  92:2  (March  1992),  449. 

^  Acme  Process  Equipment  Co.  v.  U.S.. 
171  Ct.  Cl.  324  (1965). 


1  7  Airprep  Technology,  Inc,  v.  U.S..  1 3 
FPD  18  (Fed.  Cir.  1994). 

13  Daniel  E.  Toomey,  William  B.  Fisher, 
and  Laurie  F.  Curry,  "Good  Faith  and 
Fair  Dealing:  The  Well-Nigh  Irrefragable 
Need  for  a  New  Standard  in  Public 
Contract  Law,"  Public  Contract  Law 
Journal  20:1  (Fall  1990),  87-125; 
Gonzales  v.  Freeman.  334  F.2d  570 
(1964). 

14  Phillip  M.  Kannan,  "Perkins  v.  Lukens 
Steel  Company:  Fifty-two  and 
Counting,"  Public  Contract  Law  Journal 
22:3  (Spring  1993),  463-476,  citing 
Hughes  v.  Alexandria  Scrap  Corp..  426 
U.S.  794  (1976). 

13  Toomey,  et  al. 

16  Priebe  &  Sons  v.  U.S..  332  U.S.  407 
(1947);  American  Ship  Building 
Company  v.  U.S..  228  Ct.  Cl.  220 
(1981);  ASBCA  12174,  Garritv  Corp,, 
67-2  BCA  6586,  ASBCA  9030,  Reeves 
Soundcraft  Corp..  64  BCA  6317; 

ASBCA  33,317,  Coronado  Technology. 
Inc..  88-3  BCA  20,983;  AGBCA  296, 
Robichaux  Contractors.  Inc..  72-1  BCA 
9302. 

17  Perry  v,  U.S..  294  U.S.  330,  334 
(1934).  The  first  use  of  this  defense  was 
in  Demina  v,  U.S..  1  Ct.  Cl.  190  (1865), 
in  which  a  Marine  Corps  contractor  was 
denied  recovery  for  increased  costs  under 
a  rations  contract  due  to  new  statutes 
enacted  by  Congress.  It  should  be  noted 
that  sovereign  acts  are  considered  an 
excusable  delay  under  the  FAR  Default 
clause  (see  52.249-8(c)(2),  which  allows 
acts  of  the  Government  in  either  its 


258 


sovereign  or  contractual  capacity  as  a 
basis  for  a  valid  excusable  delay). 

18  Atlas  Corn,  v.  U.S..  895  F.2d  745, 

754  (1990). 

19  Hughes  Communications  Galaxy,  Inc, 
v,  U.S.,  26  Cl.  Ct.  123,  137  (1992). 

20  Winstar  Corp.  v.  U.S..  994  F.2d  797, 
810(1993). 

21  Unites  States  Trust  Company  of  New 
York  v.  New  Jersey,  431  U.S.  1,  22 
(1976). 

22  Ibid,  22,  25. 

23  Winstar,  808;  Deming  v.  U.S.,  1  Ct. 
Cl.  190  (1865). 

24  Charles  T.  Goodsell,  The  Case  for 
Bureaucracy  (Chatham  House,  1983), 
111. 

25  Ronald  G.  Morgan,  "Identifying 
Protected  Government  Acts  under  the 
Sovereign  Acts  Doctrine:  A  Question  of 
Acts  and  Actors,"  Public  Contract  Law 
Journal  22:2  (Winter  1993),  223-233. 

26  John  Cibinic,  "Sovereignty:  Is  Our 
Government  An  Honest  Person,"  Nash  & 
Cibinic  Report  (December  1992),  191. 

22  Robert  C.  Gusman,  "The  Winstar 
Case:  A  Study  in  Abandonment,"  Federal 
Contracts  Report  (October  25,  1993), 
440. 

28  Jones  v.  U.S..  1  Ct.  Cl.  383  (1865). 

29  310  U.S.  113  (1940). 


See  Uniform  Commercial  Code, 
Section  1-102,  which  allows  the  parties 
to  establish  any  contract  terms  based  on 
commercial  practices  or  customs,  as  long 
as  the  underlying  principles  of  good  faith, 
diligence,  reasonableness,  and  care  are 
upheld. 

3 1  Quoted  in  Thomas  S.  Williams  v. 

U.S..  289  U.S.  553,  575-578  (1933). 

32  The  Federalist  Papers,  No.  81 
(Hamilton),  (New  York:  New  American 
Library,  1961),  487-488. 

33  Federal  Crop  Insurance  Corp.  v. 
Merrill.  332  U.S.  380,  383  (1947). 

34  The  Competition  in  Contracting  Act, 
with  its  bid  protest  procedures,  was  an 
explicit  waiver  of  sovereign  immunity; 
Grumman  Data  Systems  Corp.  v.  U.S., 
12FPD71  (Fed.  Cir.  1993). 

36  Ernest  Gabbard,  Charles  E. 
Rumbaugh,  and  William  A.  Shook, 
Commercial  Versus  Federal  Contracts 
(NCMA:  TIPS,  July  1992);  Kathryn 
Dean  Checchi,  "Federal  Procurement  and 
Commercial  Procurement  under  the 
U.C.C.-A  Comparison,"  Public  Contract 
Law  Journal  11:2  (June  1980),  360-378; 
Wendy  Kirby,  Commercial  Practices  for 
Commercial  Products  (Washington,  DC: 
Hogan  &  Hartson,  1988). 

36  Black  &  Decker,  Conditions  of 
Purchase  for  Tooling  Purchase  Order 
(B&D  Form  No.  10013,  2-86)  and 
Requisition  Purchase  Order  (B&D  Form 
No.  10016,  8/88);  Unisys  Corporation 
Purchase  Requisition,  Purchase  Order 
Terms  and  Conditions  (form  UC  1695, 
4/88),  Boeing  Commercial  Airplane 


259 


Company,  Purchase  Order  Terms  and 
Conditions  (D1  4100  4045,  Rev.  4/83). 

37  Steven  Kelman,  Procurement  and 
Public  Management:  Fear  of  Discretion 
and  the  Quality  of  Government 
Performance  (Washington,  DC:  AEI: 
1990),  57;  Stewart  Macaulay,  Non¬ 
contractual  Relations  in  Business:  A 
Preliminary  Study,  American 
Sociological  Review  28  (February  1963), 
55-67.  " 

38  A.L.  Haizlip,  "The  Government 
Contractor  Defense  in  Tort  Liability:  A 
Continuing  Genesis,"  Public  Contract 
Law  Journal  19:1  (Fall  1989),  116-144. 

39  Margaret  M.  Severson,  "Defense 
Industry- 1,  Injured  Parties-0:  Rights- 
Limiting  Ethical  Problems  with  Boyle 
and  the  Government  Contractor 
Defense,"  Public  Contract  Law  Journal 
21:4  (Summer  1992),  572-592. 

40  Penner  Installation  Corporation  v. 
U.S.  89  F.  Supp.  545,  547-548  (1950). 

41  Christopher  N.  Lee,  "Government 
Contracting:  A  Game-Theoretic 
Analysis,"  Public  Contract  Law  Journal 
19:2  (Winter  1990),  382-292. 

42  See,  generally,  Randy  Barnett,  ed., 
The  Rights  Retained  by  the  People 
(George  Mason  University  Press,  1989). 

43  Inventing  the  People:  The  Rise  of 
Popular  Sovereignty  in  England  and 
America  (New  York:  Norton,  1988). 

44  Stephen  L.  Collins,  From  Divine 
Cosmos  to  Sovereign  State  (New  York: 
Oxford,  1989),  168. 


45  Susan  Rose-Ackerman,  "The 
Economic  Analysis  of  Public  Law," 
European  Journal  of  Law  and  Economics 
1:1  (March  1994),  64. 

46  Morgan,  237. 

47  Herbert  Kaufman  has  demonstrated 
how  Federal  "red  tape"  develops  by 
popular  demand;  Red  Tape  (Washington, 
DC:  Brookings,  1977),  29-59. 

48  The  Nature  of  Politics  (New 
Brunswick,  NJ:  Transaction  Publishers, 
1992),  198-206. 

49  Randall  G.  Holcombe,  The  Economic 
Foundations  of  Government  (New  York 
University  Press,  1994),  168. 

50  William  E.  Kovacic,  "Regulatory 
Controls  as  Barriers  to  Entry  in 
Government  Procurement,"  Policy 
Sciences  25  (1992),  29-42. 

51  David  V.  Lamm,  "Why  Firms  Refuse 
DOD  Business:  An  Analysis  of 
Rationale,"  National  Contract 
Management  Journal  21:2  (Winter 
1988),  49. 

52  Richard  Epstein,  Bargaining  with  the 
State  (Princeton  University  Press,  1993), 
79,312. 

53  Although  this  "all  or  none" 
environment  is  the  most  common, 
another  scenario  exists  (except  when 
using  sealed  bidding):  submitting  an  offer 
containing  a  list  of  exceptions  to  the 
"boilerplate,"  since  FAR  1.4  allows 
agencies  to  grant  deviations  from  the 
regulations. 


260 


54  it  should  be  noted  that  the  value  of 
7.5  would  be  reduced  to  a  number  closer 
to  5. 1  if  the  dollar  amount  of 
Government  business  constituted  only  a 
small  fraction  of  the  firms'  total  business. 
This  is  the  case  for  Government 
purchases  of  jet  fuel;  Christopher  N.  Lee, 
Consequences  of  Constrained 
Governance:  A  Comparative  Analysis  of 
DoD  Jet  Fuel  Contracting  (George 
Mason  University,  PhD.  Dissertation, 
1990),  141. 

55  Robert  H.  Bork,  The  Antitrust 
Paradox.  A  Policy  at  War  with  Itself 
(New  York:  Basic  Books,  1978). 


261 


MULTI-VENDOR  SUPPLY  CONTRACTS  - 
A  NEW  METHODOLOGY  OF  CUSTOMER  SUPPORT 


Jeffrey  M.  Meyers,  CPCM 
Contracting  Officer 
General  Services  Administration 


ABSTRACT 

Multi-Vendor  Supply  Contracts  (MVSC)  is 
a  new  type  of  contracting  combining  the 
advantages  of  both  Multiple  Award 
Schedules  (MAS)  and  Special  Order 
Programs  (SOP).  It  permits  the  end-using 
customer  to  obtain  relatively  complex  to 
extremely  complex  commercial  products 
based  upon  their  specific  needs  and  their 
budgetary  requirements.  There  are  several 
distinctive  advantages  in  the  use  of  MVSC: 

From  the  customer’s  point  of  view,  they 
are  able  to  obtain  a  commercial  product 
from  one  of  a  myriad  of  suppliers  which 
can  meet  their  specific  needs.  In  addition 
to  the  above,  they  are  able  to  obtain  a 
product  in  a  timely  fashion  with  delivery 
time  significantly  reduced  from  “normal” 
procurements  methods  such  as  firm  fixed 
price  (FFP)  definite  quantity  buys.  They 
are  also  able  to  obtain  products  at  an 
extremely  competitive  price  -  typically 
equal  to  or  better  than  the  pricing  in  a 
definite  quantity  procurement. 

From  the  contractor’s  point  of  view,  the 
products  being  offered,  and  purchased,  are 
what  they  make  and  sell  commercially.  As 
such,  they  are  not  required  to  meet  a 
government  specification.  In  addition  to 
the  above,  the  contractor  need  not  expend 
its  resources  in  responding  to  several 
solicitations,  each  one  for  a  specified 
quantity  where  contract  award  is  based,  in 
many  cases,  strictly  on  lowest  price  with 


little  consideration  given  to  technical 
capability  or  quality. 

From  the  contracting  office’s  point  of 
view,  it  permits  a  broad  spectrum  of 
commercially  available  products  from 
which  orders  can  be  fulfilled  timely  with  a 
limited  staff.  It  also  saves  the  contracting 
office  significant  time  and  moneys  in 
avoiding  such  items  as  written  government 
specifications,  and  the  significant  costs  of 
preparing  and  issuing  solicitations, 
evaluation  of  offers,  and  contractual  award 
for  each  individual  requirement. 

The  main  strength  and  selling  point  of  the 
MVSC  program  is  its  ability  to  provide  a 
needed  product  in  a  timely  manner  at  a 
reasonable  price.  In  this  era  of  reinventing 
government,  the  MVSC  program 
streamlines  the  procurement  process  to  the 
benefit  of  the  contracting  office,  the 
contractor,  and  the  federal  government 
customer  agencies. 
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In  order  to  help  fulfill  its  mission,  the  General 
Services  Administration  (GSA)  created  a 
new,  unique,  and  innovative  form  of 
contracting  in  March  of  1991.  The  program, 
called  Standardization  and  Control  of 
Industrial-Quality  Tools  (SCIT),  is  the  first 
Multiple  Vendor  Supply  Contract  (MVSC)  in 
existence.  This  program  is  a  stock  program 
for  the  purchase  of  tools  used  for  airplane 
maintenance  and  repair. 

Since  1991,  the  use  of  Multiple  Vendor 
Supply  Contracting  has  become  more 
prevalent  due  to  the  use  of  Industrial  Funding 
and  recurring  unique  requirements  in  the 
Government's  procurement  sectors. 
Currently,  the  MVSC  program  has  extended 
into  non-stock  areas  MVSC  has  enabled  GSA 
to  better  fulfill  its  mission  through  the 
supplementation  of  its  multiple-award 
schedule  (MAS)  program. 

Congress  has  shown  how  popular  the  MAS 
program  has  become.  In  a  proposed  bill  by 
United  States  Representative  John  Conyers, 
Jr.,  the  following  opening  remarks  were 
made: 

Federal  agencies  each  year  order 
approximately  $5  billion  in  goods  and 
services  from  ’multiple  award  schedule' 
contracts  awarded  by  GSA.  This  program  is 
extremely  popular  with  Government 
contracting  officers,  because 


they  are  able  to  place  orders  from  vendor 
schedule  contracts,  sort  of  like  ordering  from 
a  Department  store  catalog,  bypassing  many 
of  the  requirements  of  the  Federal  contracting 
system.  It  has  also  become  extremely 
popular  with  schedule  contractors  as  a  result 
of  the  profits  to  be  made  from  this  'fast-track' 
method  of  Federal  contracting. 

The  above  figures  may  soon  become  a  gross 
underestimate  of  MAS  sales  volume  for 
future  years.  According  to  Title  I,  Subtitle  E, 
Part  II  -  of  the  Federal  Acquisition 
Streamlining  Act  of  1994  (The  Act),  Federal 
supply  schedules  can  now  be  used  by  State 
and  local  governments,  Puerto  Rico,  and 
Governments  of  Indian  tribes.  Section  1555 
of  The  Act  amends  subsection  201  of  the 
Federal  Property  and  Administrative  Services 
Act  of  1949  as  follows:  .  .  .  The 

Administrator  [of  GSA]  may  provide  for  the 
use  of  Federal  supply  schedules  of  the 
General  Services  Administration  by  any  of 
the  following  entities  upon  request: 

‘(i)  A  State,  any  department  or  agency  of  a 
State,  and  any  political  subdisvision  of  a 
State,  including  a  local  government. 

‘(ii)  The  Commonwealth  of  Puerto  Rico. 

‘(iii)  The  government  of  an  Indian  Tribe  .  .  . 
(Federal  Acquisition  Streamlining.  .  .) 

MVSC  is  very  similar  to  (MAS)  contracting 
in  theory  and  practice.  It  is  an  attempt  to 
solve  current  procurement  problems  through 
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the  use  of  "innovative  customer  support" 
(Schroyer). 

On  July  24,  1990,  Solicitation  Number 
FCEP-BB-90NADPB-N  was  issued  for 
Industrial  Quality  Precision-Measuring  and 
Pneumatic  Hand  Tools.  In  it,  the  Statement 
of  Work's  Objectives  state: 

This  solicitation  for  industrial  quality  tools  is 
designed  to  meet  the  following  objectives:  a. 
Establish  a  new  source  of  supply  which  will 
provide  those  customers,  actively  involved  in 
tool  standardization  and  control,  with  a 
vehicle  whereby  they  can: 

1.  Choose  Correct  items  necessary  to  be  used 
by  artisans/mechanics  in  their  performance 
of  many  various  disciplines  and  tasks 
associated  with  aircraft  maintenance  and 
aerospace  applications  in  support  of  critical 
weapon  systems. 

2.  Choose  items  to  support  activity-unique 
tool  standardization  and  control  programs 
(i.e.,  shadow  box,  vacuum  formed  tool 
cabinet  drawer  inserts,  random  inventory, 
total  accountability,  or  other  method). 

3.  Choose  compatible  replacement  items 
ensuring  continuity,  uniformity  and  economy 
in  the  standardization  and  tool  control 
program. 

b.  Maintaining  a  separate,  specialized  stock 
of  these  highly  critical  use  tools  to  provide 
supply  support  in  the  case  of  a  national 
emergency.  (Solicitation  Number 

FCEP-BB-90NADPB-N,  p.  6) 

Due  to  the  fact  that  Multiple  Vendor  Source 
Contracting  works  similarly  to  MAS 
contracting.  Federal  Acquisition  Regulation 
(FAR)  8.405-1,  entitled,  "Ordering  from 
multiple-award  schedules,"  applies  to  both 
types  of  procurements.  FAR  8.405-1  says, 


When  ordering  from  multiple-award 
schedules,  ordering  offices  shall  use  the 
procedures  set  forth  below.  When  these 
procedures  are  followed,  orders  placed 
against  schedules  will  result  in  the  lowest 
overall  cost  alternative  to  meet  the  needs  of 
the  Government. 

(a)  Orders  should  be  placed  with  the  schedule 
contractor  offering  the  lowest  delivered  price 
available.  The  ordering  office  shall  review  the 
schedule  price  lists  that  are  reasonably 
available  at  the  ordering  office.  Where  the 
ordering  office  has  available  fewer  than  three 
price  lists  from  current  schedule  contractors 
that  offer  the  required  items,  the  ordering 
activity  shall  obtain  additional  price  lists  from 
schedule  contractors  listed  in  the  GSA 
schedule  for  the  required  items.  The  ordering 
office  shall  fully  justify  in  the  contract  file 
orders  for  a  line  item  exceeding  the  price 
reasonableness  verification  threshold  at 
13.106  placed  at  other  than  the  lowest  price 
identified  in  its  review.  Justification  for 
ordering  a  higher  priced  item  may  be  based 
on  such  considerations  as  - 

(1)  Delivery  time  in  terms  of  actual  need  that 
cannot  be  met  by  a  contractor  offering  a 
lower  price; 

(2)  Specific  or  unusual  requirements  such  as 
differences  in  performance  characteristics; 

(3)  Compatibility  with  existing  equipment  or 
systems; 

(4)  Trade-in  considerations  that  favor  a 
higher  priced  item  and  produce  the  lowest  net 
cost;  and 

(5)  Special  features  of  one  item  not  provided 
by  comparable  items  that  are  required  in 
effective  program  performance.  .  .  (Federal 
Acquisition  Regulations,  Federal  Acquisition 
Change  90-5,  p.  8-11) 

The  above  justifications  still  apply  in  MVSC 
and  are  received  on  a  regular  basis.  Upon 
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occasion,  the  customer  agency  fails  to  provide 
a  justification  and  GSA  must  then  ask  the 
agency  for  the  justification  to  be  forwarded 
before  the  order  is  placed. 

The  Competition  in  Contracting  Act  (CICA) 
has  acknowledged  the  use  of  Multiple-Award 
Schedules  as  a  valid  exemption.  CICA  states, 
in  part,  that, 

.  .  .  The  term  'competitive  procedures'  means 
procedures  under  which  an  executive  agency 
enters  into  a  contract  pursuant  to  full  and 
open  competition.  Such  term  also  includes-. 
.  ."(3)  the  procedures  established  by  the 
Administrator  for  the  multiple  awards 
schedule  program  of  the  General  Services 
Administration  if- "(A)  participation  in  the 
program  has  been  open  to  all  responsible 
sources;  and 

"(B)  orders  and  contracts  under  such 
procedures  result  in  the  lowest  overall  cost 
alternative  to  meet  the  needs  of  the 
Government.  .  .  (United  States  Statutes  at 
Large.  .  .  p.  98  STAT.1180) 

In  the  use  of  MVSC  contracting,  the 
procedures  for  contractual  award  are  carried 
out  as  if  the  firm  was  submitting  an  offer  for 
an  MAS  schedule  contract.  Discount 
schedule  and  marketing  data  are  requested 
(Solicitation  FCEP-BB-90NADPB-N,  pp.  45 
-  54)  and  each  offer  is  negotiated  on  its  own 
merits  as  if  it  were  an  MAS  contract. 

On  December  22,  1987  Public  Law  100-202 
amended  Section  109  of  the  Federal  Property 
and  Administrative  Services  Act  of  1949. 
Before  this  amendment  GSA  was  only 
authorized  to  charge  the  direct  funding  of  a 
specific  procurement  to  the  ordering  agency 
(typically  used  in  a  definite  quantity 
requirements  contract)  (Loewentritt).  After 


Public  Law  100-202  was  signed,  GSA  was 
given  the  authority  to  charge  indirect  funds, 
overhead,  and  contracting.  In  essence,  GSA 
became  industrially  funded  and  the  statute 
stated  that  anything  procured  through  the 
General  Supply  Fund  could  receive  a 
mark-up  because  of  GSA's  status  as  the 
primary  contracting  office.  Until  the  use  of 
MVSC,  GSA  has  only  used  this  mark-up  on 
stock  and  special  order  contracts  (SOP) 
(Loewentritt).  The  MVSC  contracts  allow 
GSA  to  recoup  some  of  the  costs  associated  in 
establishing  these  contracts. 

MAS  allows  all  ordering  agencies  to  purchase 
directly  from  the  contractor  from  the 
contracts.  In  MVSC,  however,  the  firm 
submits  a  requisition  to  GSA  who  then 
purchases  the  product  on  the  behalf  of  the 
customer  agency.  GSA,  once  the  product  is 
delivered,  is  billed  by  the  contractor.  GSA 
then  pays  the  contractor  and  bills  the 
customer  agency  the  invoice  price  plus  a 
surcharge.  Said  surcharge  varies  by 
commodity  center  and,  in  some  cases, 
product  line  within  the  commodity  center. 

The  surcharges  stated  above  are  calculated 
based  upon  two  factors:  The  projected  sales 
of  the  Commodity  Center  and  the  projected 
costs  incurred  by  the  Commodity  Center. 
Ideally,  if  all  projections  are  accurately 
portrayed,  at  the  end  of  the  fiscal  year  the 
Commodity  Center  should  break  even. 
Typically,  the  sales  figures  or  costs  are  not 
entirely  accurate.  FCA,  for  example,  had 
projected  sales  of  $472.2  million  for  fiscal 
year  1991.  As  of  July  1991  (10  months  into 
FY  1991)  the  center  had  sales  of  $604.4 
million  (Feit  Memorandum).  At  the  same 
time,  6FE-CO  had  projected  sales  of  $46.7 
million  and  only  achieved  $27.6  million  for 
the  same  time  period  (Feit  Memorandum). 
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The  revenue  of  FCA,  based  upon  the  above, 
is  $2.6  million  above  what  was  anticipated, 
however  6FE-CO  is  $1.1  million  lower  than 
anticipated  (Feit  Memorandum). 

MVSC's  primary  area  of  use  is  in  the 
fulfillment  of  unique  requirements 
(Schroyer).  It  is  not  designed  to  replace,  but 
rather,  to  supplement  the  MAS  contracts.  In 
addition  to  the  above,  the  main  purpose  of 
MVSC  is  not  to  make  money  for  GSA,  but  to 
support  GSA's  customers.  Mr.  Gormley 
emphatically  states  that,  "Industrial  Funding 
is  not  the  issue  -  efficient  customer  support  is 
the  issue"  (Gormley).  This  attitude  is 
re-emphasized  by  Larry  Schroyer  who  states, 
"The  primary  reason  is  not  to  make  money 
but  to  increase  services  [to  the  using 
agencies]"  (Schroyer).  The  theory  behind 
MVSC  is  that  if  one  is  able  to  meet  and 
exceed  customer  expectations  and  needs,  then 
the  customer  will  be  more  willing  to  use  this 
non-mandatory  source  in  the  procurement  of 
necessary  equipment. 

The  advantages  in  the  use  of  MVSC  are 
significant.  It  allows  the  use  of  standard 
commercial  products  to  fulfill  an  agency's 
need  wherever  possible.  This  is  significant  in 
reducing  the  cost  of  writing  and  maintaining 
specifications,  first  article  testing,  and, 
perhaps  most  importantly,  ease  of 
contracting. 

Historically,  the  government  has 
acknowledged  the  use  of  MAS  contracting 
based  upon  the  theory  that  if  a  firm  makes 
and  sells  a  significant  number  of  a  product  to 
the  general  public  and  the  sales  are  from  a 
commercial  pricelist,  then  it  is  assumed  that 
the  firm  makes  a  quality  product.  If  such 
were  not  the  case,  the  economic  theories  of 


supply  and  demand  would  dictate  that  the 
firm  would  go  out  of  business. 

The  use  of  MVSC  allows  each  specific 
agency  to  determine  exactly  which  item,  by 
brand  name  and  model,  will  best  meet  its 
unique  requirements.  In  addition  to  the 
above,  it  is  required  that  the  firm  offer  its 
standard  commercial  warranty  (Schroyer).  In 
the  past,  if  a  product  was  defective  but  still 
met  government  specifications,  the  firm  was 
not  required  to  replace  the  product 
(Schroyer).  The  use  of  this  warranty 

provision  in  SCIT  is  extremely  important 
since  the  buying  agency  is  assured  of  the 
quality  of  the  product. 

The  use  of  MVSC  also  allows  for  better 
customer  support.  In  the  past,  6FE-CO,  for 
example,  could  not  guarantee  the  same 
quality  product  each  and  every  time  the  using 
agency  had  a  requirement.  With  the  use  of 
SCIT,  the  using  agency  has  more  assurances. 
Since  the  contractors  can  make  and  sell  their 
own  specific  products  instead  of  meeting  a 
government  specification,  back-orders  and 
additional  costs  associated  with  first  article 
testing  and  price  analysis  are  avoided.  FAR 
15.804-3  states: 

Exemptions  from  or  waiver  of  submission  of 
certified  cost  or  pricing  data. 

(a)  General.  Except  as  provided  in 
paragraphs  (b)  and  (c)  below,  the  contracting 
officer  shall  not  require  submission  or 
certification  of  cost  or  pricing  data  when  the 
contracting  officer  determines  that  prices  are  - 

(1)  Based  on  adequate  price  competition  (see 
paragraph  (b)  below); 

(2)  Based  on  established  catalog  or  market 
prices  of  commercial  items  sold  in  substantial 
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quantities  to  the  general  public  (see  paragraph 

(c)  below);  or 

(3)  Set  by  law  or  regulation  (see  paragraph 

(d)  below).  .  .  (Federal  Acquisition 

Regulation  15.804-3,  p.  246) 

Since  the  items  being  offered  are  commercial 
off-the-shelf  items,  it  seems  likely  that  shorter 
delivery  times  are  reasonable.  In  the  SCIT 
program,  for  example,  no  new  product  run  is 
required.  This  particularly  applies  to  other, 
more  complex  items  which  are  planned  to  be 
contracted  using  MVSC.  Generally  speaking, 
the  more  complex  the  item,  the  longer  it  takes 
to  make  according  to  a  government 
specification.  If,  on  the  other  hand,  the 
vendor  is  able  to  make  one  of  their  standard 
items  and  use  it  to  meet  the  government's 
needs  we  avoid  government  unique 
requirements  and  they  can  fulfill  the 
government's  order  in  its  routine  daily 
operations. 

Although  the  advantages  of  better  products, 
avoidance  of  specification  risk,  better 
customer  support,  and  shorter  time  of 
delivery  are  significant,  there  are  also  some 
very  real  pitfalls.  The  first  major  danger  is 
lack  of  vendor  interest.  If  the  government 
attempts  to  make  a  MSVC  program  for  a 
commodity  and  they  are  unable  to  obtain 
adequate  coverage,  then  the  program  is 
doomed  for  failure  (Hegarty).  To  illustrate, 
if,  using  SCIT  as  an  example,  DOD  has  ten 
basic  brands  of  tools  they  acquire  from  ten 
different  contractors,  and  only  two  of  those 
contractors  submit  offers,  the  proportionate 
amount  of  DOD  moneys,  hence  GSA  support 
to  DOD,  will  not  be  spent  under  SCIT. 

SCIT  is  a  stock  program.  In  anticipation  of 
forthcoming  orders,  GSA  has  spent  $6 
million  in  the  stocking  of  products 


(Schroyer).  This  was  done  in  order  to  avoid 
such  areas  as  backorders  and  non-timely 
delivery.  One  of  the  considerations  (in 
determining  how  many  products  to  stock)  was 
an  expected  annual  purchase  volume.  If  GSA 
has  grossly  over  or  under  estimated  the  sales 
volume  then  they  may  wind  up  losing  an 
important  customer  base. 

Another  pitfall  would  be  encountered  if  the 
program  should  ever  obtain  a  poor  reputation. 

If,  after  its  inception,  things  do  not  run  as 
smoothly  as  hoped  for,  and  either  firms  or 
using  agencies  cannot  obtain  what  is  required 
from  GSA,  then  there  is  the  very  real 
possibility  that  GSA  will  be  unable  to  market 
the  program  in  the  future. 

Another  major  consideration  is  the  potential 
abuse  of  agencies  in  their  requisitions.  The 
current  mode  of  contracting  starts  when  the 
agency  sends  a  requisition  to  GSA  stating 
specifically  which  item  is  required.  The 
agency  is  required  to  document  their  file  in  an 
effort  to  justify  why  the  cheapest  item  was 
not  procured.  While  the  justification  is  not  a 
major  issue  for  the  SCIT  program,  it  may 
become  an  issue  for  other  MVSC  contracts. 
In  essence,  it  can  lead  to  sole-source  buys. 
GSA,  the  ordering  agency,  original  did  not 
review  the  justification.  In  the  past,  an  agency 
can  simply  make  a  requisition  for  a  more 
expensive  item  and  have  an  inaccurate 
justification  in  their  files  (Hegarty).  This  has 
been  rectified  by  the  requisitioning  office 
forwarding  a  copy  of  the  justification  to  GSA. 

If  GSA,  as  the  ordering  agency,  needs 
further  clarification  it  may  then  obtain  it  from 
the  customer  agency. 

MVSC  is  questionable  in  regards  to  CICA. 
While  it  is  similar  to  MAS  contracting,  it  is 
not  one  of  the  five  recognized  forms  of 
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competitive  procedures  (Hegarty).  While  it  is 
not  illegal  per  se,  it  seems  to  delve  into  the 
gray  area  of  legality.  It  can  be  argued  that 
since  MSVC  is  based  upon  the  same 
principles  as  MAS  contracting,  it  is  indirectly 
recognized  as  a  competitive  procedure. 

Another  problem  with  MVSC  contracting  is 
the  fixed-rate  surcharges.  In  theory,  all  GSA 
has  to  do  once  a  requisition  has  been  received 
is  to  cut  a  delivery  order  to  the 
vendor/warehouse  and  the  product  will  be 
forwarded  to  the  end-user.  Since  the  cost 
associated  with  the  delivery  order  is 
primarily  the  same  for  any  size  order,  then 
one  may  expect  there  to  be  a  flat  processing 
fee  plus  a  substantially  reduced  fee  for  each 
tool,  instead  of  a  surcharge  on  the  total  order. 

The  use  of  the  latter  formula  would 
encourage  consolidation  of  requirements  as 
well  as  a  more  reasonable  savings  for  higher 
dollar  volume  orders  (Hegarty). 

This  system  may  be  used  in  the  future,  but  it 
seems  too  complex  for  today's  use.  GSA  has 
difficulty  in  accurately  projecting  the  total 
dollar  volume  of  orders  received  per  annum. 
It  is  unreasonable  to  expect  them  to  be  able  to 
accurately  project  the  number  of  individual 
purchase  orders,  the  dollar  volume  of  each 
order,  and  from  that  to  calculate  a  rate  which 
would  support  a  break-even  fiscal  year 
surcharge. 

The  final  complication  to  consider  is  price. 
Historically,  competition  is  a  great  influence 
in  pricing.  SCIT's  prices  for  similar 
equipment  are,  surprisingly,  higher  than  those 
under  the  normal  stock  contracts  for  tools. 
This  was  justified  based  upon  the  differing 
terms  and  conditions,  such  as  warranty  and 
marking,  as  well  as  the  criteria  of  award 
being  different.  SCIT  avoids  direct 


competition  and  the  MAS-type  of  award 
criteria  (a  price  equal  to  or  greater  than  the 
most  favored  customer)  were  used 
(Schroyer).  Due  to  the  different  form  of 
contracting,  combined  with  different  terms 
and  conditions,  the  higher  prices  were  found 
fair  and  reasonable. 

The  Automotive  Commodity  Center  has 
implemented  the  MVSC  program  for  three 
areas:  fire  fighting  vehicles  (entitled  ALF  -  A 
Lotta  Firetrucks),  waste  disposal 
vehicles(entitled  TRACY  -  Trash  collection 
and  Recycling  equipment  Available 
Commercially  for  You),  and  tankers  and 
refueling  vehicles  (entitled  NATE  -  Now 
Available  Tankers  and  Equipment  servicing 
vehicles)(Feit).  The  successful  use  and 
implementation  of  SCIT  was  the  deciding 
factor  in  the  use  of  MVSC.  In  spite  of  the 
same  basic  concept,  there  are  some  major 
differences.  Fire  fighting  vehicles,  waste 
disposal  vehicles  and  tankers  and  refueling 
vehicles  are  items  of  a  very  complex  nature 
where  tools  are,  by  comparison,  simple. 

The  above  programs  require  an  extreme 
amount  of  contract  administration,  technical 
support,  and  both  vendor  and  agency  interest 
in  order  for  them  maintain  their  usefulness. 
Another  major  difference  is  the  fact  that  the 
SCIT  program  was  a  Stock  based  program 
where  fire  fighting  vehicles,  waste  disposal 
vehicles,  and  tankers  and  refueling  vehicles 
are  purchased  when  the  requisitions  are 
received  from  the  customer  agency.  The 
above  has  resulted  in  some  slightly  different 
clauses  than  those  which  appear  in  the  SCIT 
solicitation  or  any  MAS  solicitation. 

A  final  area  which  has  evolved  out  of  the 
MVSC  program  is  the  use  of  this  type  of 
contracting  for  forklift  trucks  and  off-road 
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vehicles  (Theunissen).  GSA’s  Fort  Worth, 
Texas  offices  have  implemented  a  program 
for  both  forklift  trucks  and  off-road  vehicles. 

The  Automotive  Commodity  Center 
originally  assisted  in  the  implementation  of 
the  forklift  truck  program  by  providing 
technical  support  in  the  preparation  and  initial 
implementation  of  the  program. 

The  use  of  MVSC  in  the  area  of  complex 
equipment  purchases  such  as  the  ALF  or 
TRACY  programs  has  become  a  very 
effective  contacting  approach.  ALF  and 
TRACY  were  implemented  in  July  1992. 
Prior  to  ALF  and  TRACY 

Firetrucks  are  never  generic.  Although  each 
truck  is  based  on  the  same  general 
body/chassis,  there  are  literally  thousands  of 
specialized  combinations  of  options  and 
accessories  which  can  be  added  to  meet  each 
agency's  individual  requirements.  On  this 
basis,  MVSC  contracting  seems  to  be  the 
most  promising  form  of  contracting.  It  will 
aid  GSA  in  supplying  a  viable  product  to  the 
end-user  agencies.  This  logic  also  applies  to 
refueling  vehicles. 

Using  MVSC  contracting  in  the  above 
situations  has  significantly  reduced  contract 
lead  times,  delivery  time,  and  cost.  This  is 
because  GSA  in  the  past,  purchased  the 
above  equipment  on  a  definite  quantity  basis 
(Feit).  Each  individual  requisition  must  be 
synopsized,  negotiated,  and  awarded  before 
delivery  can  take  place.  This  is  inefficient 
with  regards  to  time,  as  it  takes,  on  the 
average,  two  and  a  half  to  three  years  from 
receipt  of  requisition  to  delivery  of  the  final 
product. 

In  the  past,  before  MVSC  was  used  in  the 
Automotive  Commodity  Center,  GSA 


received  and  fulfilled  approximately  five 
requisitions  for  fire  fighting  vehicles  and  five 
requisitions  for  waste  disposal  vehicles.  The 
total  number  of  vehicles  per  year  ranged 
between  five  to  ten  vehicles  per  annum  based 
upon  the  above  requirements. 

From  July  1,  1992,  the  inception  of  the  ALF 
and  TRACY  programs,  to  December  31, 
1994,  a  two  and  a  half  year  period,  GSA  has 
received  and  processed  165  requisitions  for  a 
total  of  206  vehicles  under  ALF  and  161 
requisitions  for  204  vehicles  under  TRACY. 

The  dollar  value  of  the  orders  placed  under 
ALF,  over  the  time  period  stated,  are  over 
$36  million  and  sales  under  TRACY  exceed 
$18  million.  The  surcharge  received  from 
the  above  sales,  combined,  over  said  period 
exceed  $850,000.00. 

Previously,  it  was  stated  that  one  of  the  goals 
of  MVSC  was  to  increase  customer  support 
as  well  as  to  provide  products  delivered  on  a 
timely  basis.  It  was  also  stated  that  it  was  not 
unusual  for  vehicles  previously  purchased 
under  a  definite  quantity  procurement  using  a 
federal  specification,  to  take  two  and  a  half  to 
three  years  from  requisition  receipt  to 
delivery  of  a  vehicle.  Of  the  206  vehicles 
ordered  under  ALF  148  have  been  delivered. 
Similarly,  of  the  204  vehicles  ordered  under 
TRACY,  174  have  been  delivered. 

On  average,  the  delivery  time  under  the 
above  programs  ranges  between  six  months  to 
one  year,  dependent  upon  the  particular 
vehicle  being  purchased  and  the  contractor 
cited  in  the  requisition. 

The  Automotive  Commodity  Center  has, 
upon  occasion,  even  been  able  to  assist 
customers  when  an  urgent  need  occurred.  In 
such  instances  vehicles  have  been  provided  in 
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a  matter  of  days  to  weeks.  GSA,  in  such 
instances,  has  contacted  each  contractor  to  see 
if  any  vehicles  are  currently  in  production  or 
have  just  come  off  of  the  assembly  line  where 
the  original  customer  who  ordered  it  has  been 
unable  to  fulfill  their  obligation  to  buy  said 
vehicle. 

If  the  model  produced  is  under  the  GSA 
contract  we  can  then  procure  the  vehicle  and 
provide  it  in  an  expedited  manner  to  the 
customer  agency. 

Hopefully,  this  paper  has  demonstrated  the 
effectiveness  of  MVSC  as  an  alternative 
source  of  customer  support  for  Federal 
Agencies.  The  program  has  exceeded  the 
expectations  of  GSA  in  several  key  areas  as 
well  as  providing  a  new  way  of  providing  the 
customer  what  can  best  meet  his  individual 
needs. 

It  demonstrates  GSA’s  commitment  to  the 
customer  as  well  as  to  their  attitude  and  belief 
of  procuring  better,  cheaper,  faster,  or  not  at 
all. 

As  a  final  analysis,  the  use  of  Multiple 
Vendor  Source  Contracting  will  become  more 
prevalent  in  the  future  due  to  the  use  of 
industrial  funding  in  the  government's 
procurement  sectors.  This  form  of 
contracting  is  designed  specifically  to  address 
areas  which  are  not  adequately  being  covered 
by  the  more  conventional  and  traditional 
forms  of  contracting.  While  there  are  some 
difficulties  in  establishing  the  contracts,  some 
dangers,  including  extreme  cost  risk,  may 
exist.  The  advantages  quickly  outweigh  the 
disadvantages.  MVSC  contracts  will  become 
more  prevalent  in  very  complex 
commercial-type  equipment  where  the 
contracting  officer  must  be  in  contact  with  the 
firms  on  an  almost  daily  basis.  MVSC  will 


become  more  prevalent  not  only  because 
GSA  is  industrially  funded,  but  also  because 
it  is  an  innovative  approach  to  help  meet 
GSA's  customers'  requirements  in  what  is 
currently  the  most  inexpensive  and 
cost-effective  manner.  The  program  is 
specifically  designed  to  stream-line  the 
procurement  process  for  unique,  but 
recurring,  requirements.  The  tremendous 
success  of  the  Pilot  Program  of  MVSC  can  be 
attributed  to  forethought  and  painstaking 
thoroughness. 
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ABSTRACT 

The  High  Probability  of  Intercept  (HPI) 
Receiver  System  provides  emitter  intercept, 
signal  processing,  and  emitter  identification 
of  simple  and  complex  radar  emissions 
throughout  the  Super  High  Frequency  (SHF) 
and  Extremely  High  Frequency  (EHF) 
spectrum  and  will  be  installed  on  SSN  688 
Class  and  possibly  SSN  21  Class  submarines. 
The  HPI  makes  significant  use  of 
Commercial  Off-the- Shelf  (COTS)  items 
which  were  repackaged  to  meet  the 
submarine  electronic  warfare  usage 
requirements,  resulting  in  a  “ruggedized” 
COTS  system.  HPI  has  successfully 
completed  its  First  Article  Qualification  Test, 
which  included  the  submarine  operational 
environments  of  shock,  vibration,  airborne 
and  structureborne  noise,  operating  and  non¬ 
operating  temperature,  and  electromagnetic 
compatibility.  The  HPI  uses  standard 
Original  Equipment  Manufacturer  (OEM) 
subassemblies,  such  as  Intel  486-33/66 
mother  board,  Planar,  Inc.  digital  plasma 
display.  Motorola  68020  and  68040  Signal 
Processor  circuit  boards,  COTS  removable 
hard  disk  drive,  and  off-the-shelf  microwave 
components  and  solid  state  amplifiers.  The 
HPI  design  was  specifically  structured  to 
support  survivability  and  reduce  stress  on  the 
COTS  items.  In  addition,  extensive 
reliability  tracking  and  quality  vendor 
selection  efforts  were  implemented  to  ensure 
that  COTS  failure  problems  were  quickly 
identified,  analyzed,  and  remedied  by  the 
vendors  and  that  subquality  vendors  were 


expeditiously  eliminated  from  the  qualified 
vendors  list.  Further,  Environmental  Stress 
Screening  (ESS),  reliability  burn-in,  and 
reliability  growth  test  programs  were 
implemented  to  assist  in  the  identification  of 
problem  vendors  and  COTS  manufacturing 
process  and  component  selection 
deficiencies.  HPI  is  in  production  and  is  well 
liked  and  easily  operated  by  Fleet  operators. 

INTRODUCTION 

Background.  On  29  June  1994,  Secretary 
of  Defense  William  Perry  issued  a  directive 
to  increase  access  to  commercial  state-of- 
the-art  technology  and  promote  the 
development  of  dual-use  commercial  and 
military  processes  and  products  in  order  to 
expand  the  industrial  base  that  is  capable  of 
meeting  defense  needs  at  lower  costs.  1/ 
The  use  of  COTS  in  military  systems  needs 
to  be  carefully  reviewed  to  ensure  that  the 
operational  requirements  can  be  achieved 
within  the  program’s  cost  and  schedule 
constraints.  A  significant  requirement  is  the 
operational  use  and  mission  environments, 
such  as  shock,  vibration,  temperature,  noise, 
and  electromagnetic  compatibility. 

Scope.  This  paper  provides  an  example  of 
how  COTS  items  can  be  effectively  used  in  a 
military  system  by  repackaging  techniques  to 
meet  the  submarine  electronic  warfare  usage 
requirements,  resulting  in  a  “ruggedized” 
COTS  system.  Other  COTS  lessons  learned 
are  discussed. 
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COTS  APPROACH 

The  HPI  Receiver  System  covers  the  SHF 
and  EHF  frequency  spectrum.  The  system 
includes  a  Radio  Frequency  (RF)  distribution 
and  amplifier  unit  (RFDU),  a  receiver 
processor  unit  (RPU),  and  a  system  control 
display  processor  unit  (CDPU).  The  system 
provides  emitter  intercept,  signal  processing 
and  emitter  identification  of  simple  and 
complex  radar  emissions  throughout  the 
SHF/EF1F  region.  Figure  1  provides  a 
picture  of  the  HPI  Receiver  System. 

The  SHF/EHF  HPI  makes  considerable  use 
of  COTS  items.  The  CDPU  uses  a  standard 
OEM  Intel  486-33/66  mother  board  with 
integral  processor  and  RAM  memory  and 
standard  off-the-shelf  Planar,  Inc.  plasma 
display.  A  COTS  removable  hard  disk  drive 
was  environmentally  mounted  in  HPI  to  meet 
shock  and  other  requirements.  The  RPU 
uses  two  68020  and  one  68040  Motorola 
CPU  Non-Developmental  Item  (NDI)/COTS 
circuit  boards.  The  RFDU  makes  extensive 
use  of  off-the-shelf  NDI  passive  microwave 
components  and  solid  state  amplifiers.  The 
HPI  signal  processing  software  makes 
extensive  use  of  previously  developed 
Government  software  from  the  Navy’s 
AN/ALR-67  system  and  the  commercially 
available  real  time  UNIX  multi-tasking 
operating  system  and  utilities.  In  addition, 
the  signal  processor  design  is  an  open  system 
architecture  which  uses  a  Versatile  Module 
Eurocard  (VME)  bus  and  standard  OEM 
card  cage  to  allow  for  easy  expansion  of 
processing  and  interface  capabilities.  The 
processor  to  controller  interface  uses  a 
standard  IEEE  Ethernet  bi-directional  data 
link. 

In  order  to  meet  the  stringent  qualification 
and  operational  requirements  imposed  by  the 
submarine  operating  environment  for 


airborne/ structureborne  noise,  operating  and 
non-operating  temperature,  shock,  vibration, 
and  electromagnetic  compatibility,  the  HPI 
unit  designs  were  specifically  structured  to 
support  survivability,  and  where  applicable, 
reduce  stress  on  the  COTS  items.  This  is  a 
significant  design  requirement  that  affects  the 
COTS  reliability  and  has  been  demonstrated 
during  the  environmental  qualification 
testing.  The  COTS  items  that  have  sufficient 
design  margins  for  the  electrical,  mechanical, 
thermal,  and  structural  design  requirements, 
as  shown  in  design  analyses  and 
demonstrated  by  actual  test  results,  will 
generally  result  in  a  “robust  design”  that  will 
meet  the  mission  reliability  and  performance 
requirements  of  the  system.  Sufficient 
design  margins  are  critical  for  mission  critical 
items,  such  as  the  HPI  system  which  has  a 
radar  threat  warning  requirement,  that  must 
exhibit  high  reliability.  High  reliability  will 
not  only  result  in  achievement  of  mission 
success  but  will  also  result  in  significant 
reductions  in  corrective  maintenance  and  life 
cycle  operating  and  support  costs.  Since 
military  systems  operate  in  hostile 
environments  with  extreme  temperatures  and 
mechanical  stresses,  reliability  is  crucial  to 
the  success  of  the  mission  or  the  system’s  life 
cycle.  2/ 

In  addition,  extensive  reliability  tracking  and 
quality  vendor  selection  efforts  were 
implemented  in  HPI  to  assure  that  COTS 
failure  problems  were  quickly  identified, 
analyzed,  and  remedied  by  the  vendors  and 
that  subquality  vendors  were  expeditiously 
eliminated  from  the  qualified  vendors  list. 
Environmental  Stress  Screening  (ESS), 
reliability  burn-in,  and  reliability  growth  test 
programs  were  also  implemented  to  assist  in 
the  identification  of  infant  mortality  failure 
trends  and  manufacturing  process  and 
component  selection  deficiencies  in  COTS 
items. 
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The  HPI  system  design  provided  for  future 
upgrades  of  the  COTS  items  which  are 
interchangeable  to  satisfy  the  HPI 
supportability  requirements.  The 

supportability  of  these  COTS  items  will  need 
to  be  continually  addressed  as  this  is  a 
significant  issue  due  to  the  rapid 
obsolescence  of  COTS  items.  The  use  of 
VME  backplanes  and  VME  compatible 
selection  of  interfaces  and  designer  boards 
will  reduce  the  operational  and  supportability 
impacts  caused  by  component  obsolescence, 
which  can  be  expected  in  the  dynamic  COTS 
industry. 

SOLICITATION 

The  HPI  specification  was  a  full 
performance,  competitive,  firm  fixed-price 
contract  that  was  tailored  to  eliminate  any 
unnecessary  Military  Specification  or 
Standard  imposed  requirements  without 
sacrifice  of  submarine  operational 
environmental  qualifications.  The 
Government  maintained  a  close  working 
relationship  with  the  contractor  to  ensure 
that  the  Government  received  the  best 
quality  system  for  the  best  price.  Quality  and 
performance,  even  to  the  component  level, 
were  emphasized  in  this  procurement.  The 
working  relationship,  fostered  through 
monthly  in-process  status  and  technical 
reviews  between  the  Government  and  the 
contractor,  allowed  for  early  identification  of 
problems,  effective  evaluation  of 
Government  and  contractor  engineering  and 
logistic  recommendations,  and  expeditious 
resolution  of  program  impacts  within  the 
framework  of  the  fixed  price  contract. 

LESSONS  LEARNED 

The  following  summarizes  the  lessons 
learned: 


1.  Get  operator  input.  The  HPI  system  was 
tested  by  Navy  experts  from  the  Fleet,  i.e., 
personnel  who  would  operate  the  HPI 
system  on  a  daily  basis.  By  early 
involvement  of  the  Fleet  operators,  the 
design  activity  was  able  to  identify 
operational  deficiencies  prior  to  deployment. 

2.  Maintain  a  working  relationship  between 
engineering  and  logistics  disciplines.  This 
helps  each  specialist  understand  the  “system 
picture”  and  facilitates  comprehension  of 
technical  and  logistics  interrelations. 

3.  Use  a  team  approach  which  involves  not 
only  a  close  working  relationship  between 
Government  and  contractor  engineers  and 
logisticians  but  also  other  relevant 
Government  activities,  e.g.,  Government 
depot,  In-Service  Engineering  Agent, 
installing  activities,  and  Inventory  Control 
Point. 

4.  Structure  the  system  design  to  reduce 
stress  on  COTS  items.  High  reliability  can 
be  achieved  in  a  system  with  COTS  items 
when  the  system  design  is  structured  to 
support  survivability  and  reduce  stress  on  the 
COTS  items  where  applicable.  This  must  be 
demonstrated  during  the  environmental 
qualification  testing. 

5.  Maintain  close  engineering  oversight  of 
COTS  vendors’  quality  since  vendor 
manufacturing  quality  can  differ  significantly 
between  vendors  for  the  same  form,  fit  and 
functionally  equivalent  parts  and  assemblies. 

6.  Pay  attention  to  operational  Human 
Machine  Interface  (HMI)  and  Human 
Factors  which  have  a  significant,  beneficial 
impact  on  operational  training.  In  this  case, 
a  scheduled  two  day  basic  operational 
training  course  was  reduced  to  a  half  day, 
and  a  ten  day  operations  and  maintenance 
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training  course  was  reduced  to  seven  days 
due  to  good  HMI. 

7.  Plan  for  future  upgrades  of  COTS  items. 
A  system  with  COTS  items  must  provide  for 
future  upgrades  of  the  COTS  items  which 
must  be  interchangeable  to  satisfy  the 
supportability  requirements  of  the  system. 
The  use  of  VME  backplanes  and  VME 
compatible  interfaces  and  circuit  boards  will 
reduce  the  impact  on  the  system  design 
performance  caused  by  component 
obsolescence. 

8.  Pay  careful  attention  to  and  streamline  the 

performance  specification  and 

documentation  requirements  which  can 
facilitate  the  use  of  COTS  in  military  systems 
and  enable  the  achievement  of  stringent 
performance  requirements.  The  design 
activity  must  have  highly  competent 
technical  and  logistics  expertise  to 
successfully  accomplish  streamlining  of  the 
requirements. 

9.  Use  the  vendor  and  sub-vendors’ 
technical  documentation  package  to  the 
maximum  extent  possible.  If  the  vendor’s 
documentation  package  produces  adequate 
information,  or  if  it  can  be  made  complete 
with  minor  revisions,  then  specifying  “Navy 
standard”  data  item  requirements  for  format 
compliance  or  consistency  is  non-productive. 

10.  Use  an  open  system  architecture  design. 
The  use  of  an  open  system  architecture  is  an 
absolute  must  when  extensive  use  of  COTS 
computer  technology  is  used.  With  the  rapid 
pace  of  advancement  in  the  OEM  computer 
industry,  the  system  must  be  easily  upgraded 
with  new  commercially  available  technology 
products  and  processes  or  the  system  will 
soon  be  overcome  by  obsolete  part  redesign. 


CONCLUSION 

The  Government-Industry  working 
relationship  resulted  in  a  successful  First 
Article  Test  of  the  HPI  system.  It  also 
provided  an  improvement  to  the  contractor’s 
product  line.  The  HPI  system  is  a  system 
with  “ruggedized  “  COTS  items  that  meets 
the  required  form,  fit  and  function  of  the 
performance  specification.  This  system  is 
well  liked  and  easily  operated  by  Fleet 
operators.  The  stringent  environmental 
design  requirements  can  be  achieved  by  the 
COTS  items  by  careful  attention  to  the 
system  design  architecture  that  must  be 
structured  to  support  survivability  and 
reduce  stress  on  the  COTS  items.  The 
COTS  design  margins  for  high  reliability 
must  be  verified  in  design  analyses  and 
demonstrated  by  actual  environmental  test 
results  in  order  to  produce  a  “robust  design” 
that  will  meet  the  mission  reliability  and 
performance  requirements  of  the  system. 
Close  engineering  oversight  of  COTS 
vendors’  quality  must  be  maintained  since 
vendor  manufacturing  quality  can  differ 
significantly  between  vendors.  The  use  of  an 
open  system  architecture  is  critical  when 
COTS  computer  technology  is  used  to  allow 
for  future  upgrades  with  new  commercially 
available  products  and  processes. 
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MULTIPLE  AWARD  REQUIREMENTS  CONTRACTING 


Mark  Werfel 


L  ABSTRACT 

A  new  contracting  approach  for  information 
systems  products  available  in  the  commercial 
marketplace  is  required.  Based  on  a  multiple 
award  approach,  after  an  initial  competition,  a 
few  select  firms  would  be  allowed  to  offer  a 
broad  range  of  products  at  prices  which  they 
set,  but  would  be  required  to  meet 
contractually  established  standards  or  other 
requirements.  These  products  and  prices  would 
be  offerred  in  electronic  catalogues.  Users 
would  be  free  to  select  the  best  value  from 
what  is  offerred  when  they  review  these 
catalogues,  and  reflect  their  satisfaction  with 
these  firms,  the  products  they  offer,  and  the 
support  receivedwhen  they  place  additional 
orders,  making  past  performance  more 
effective  than  ever  before.  These 
improvements  apply  Government  wide,  and 
may  also  be  used  for  other  commercial 
products  when  a  large  business  volume,  rapid 
product  improvements  and  turbulent  pricing 
exist. 

A.  Problem  Statement, 

The  benefits  of  competitively  set  prices  do  not 
persist  after  award.  Currently,  contracts  for 
these  products  take  several  years  to 
competitively  award,  but  due  to  the  changing 
nature  of  the  marketplace  (offering  rapid 
technologic  improvements  and  as  rapid  price 
reductions),  the  products  offered  at  award  at 
competitively  set  prices  are  normally  updated 
using  technology  refreshment  clause 


provisions.  However,  now  that  pricing  is 
noncompetitive,  the  prices  are  not  discounted 
to  the  extent  reflected  in  the  initial  award. 

The  Government  accepts  the  loss  of  those 
discounts  because  of  the  inordinate  amount  of 
time  required  for  and  the  difficulties  associated 
with  introducing  a  new  prime  contractor  (the 
National  Performance  Review  cites  4  years  for 
Government  vs  13  months  for  Industry).  This 
condition  is  well  known  to  both  parties,  and  for 
that  reason,  the  result  is  the  loss  of  effective 
control  over  a  single  prime  contractor  after 
award  including  the  lack  of  an  effective 
measure  for  introducing  and  pricing  new  or 
improved  products.  In  effect,  competition's 
benefits,  as  currently  practiced,  only  are  present 
for  the  brief  period  immediately  after  contract 
award  (and  may  not  be  then,  if  initial  testing  or 
administrative  delays  defer  ordering,  in  a 
market  which  characteristically  offers 
significant  and  almost  constant  price  drops). 

B.  Solution  Overview. 

Instead,  the  suggestion  would  make 
competition  effective  and  persistent,  by 
awarding  "Multiple  Award  Requirements 
Contracts"  (MARCs).  As  suggested  by 
"Multiple  Award",  several  contractors  would 
be  awarded  prime  MARC  contracts,  and 
remain  in  competition  with  one  another 
throughout  contract  life.  As  suggested  by 
"Requirements  Contracts,"  defined  users  would 
have  to  go  to  one  or  more  of  the  awardees  for 
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defined  products  (the  Approach  section 
describes  this  in  detail). 

Briefly  the  process  would  work  like  this: 

Periodically,  MARC  contractors  would 
unilaterally  update  their  offerings,  striving  for 
business  on  that  and  on  a  past  performance 
basis. 

Due  to  the  commercial  nature  of  these 
products,  more  reliance  would  be  placed  on 
contractors,  and  less  Government  involvement 
and  oversight  would  be  required. 

MARC  contractors  would  be  responsible  for 
the  products  they  provide  whether  they 
produce  them  or  their  vendors  do;  and  for 
overall  customer  support  and  satisfaction. 
Contractors  failing  to  support  users  would 
simply  not  get  future  orders,  because  users 
would  issue  new  orders  to  the  other  MARC 
contractors  (and  contracting  personnel  would 
ensure  appropriate  action  was  taken  on  the 
existing  problem  orders). 

Users  would  be  accountable  for  the  products 
they  select,  and  the  prices,  because  their  access 
to  a  more  competitive  marketplace  would  be 
direct  and  unrestricted. 

Electronic  media  would  be  used  to  accomplish 
contract  updates  instead  of  the  less  productive 
and  highly  people  intensive  mechanics  currently 
used. 

Expectating  that  MARC  contractors  would 
offer  better  deals  to  the  Government  than 
otherwise  available,  if  disappointed,  users 
would  not  be  absolutely  required  to  go  to 


them,  but  would  identify  when  and  why  they 
did  not. 

H.  BACKGROUND 

A.  Current  requirements  contract  process. 

Currently,  these  may  be  competed,  promising 
the  successful  bidder  all  of  the  work  of  an 
identified  type  of  business  that  an  identified  set 
of  users  will  place  over  a  defined  period. 
Source  selection  criteria  would  normally 
include  factors  which  measure  the  cost  and  also 
the  value  to  the  Government  of  various 
proposed  solutions.  Associated  terms  and 
conditions  are  generally  set  by  the  Government 
in  the  solicitation,  and  generally  apply  to  all 
bidders.  Once  awarded,  work  is  ordered  as 
established  by  that  process.  The  awardee  has 
the  right  to  sue  the  Government  for  breach  if 
orders  are  placed  elsewhere,  even  if  a  better 
deal  becomes  available.  However,  if  the 
requirements  contractor's  prices  become  higher 
than  market  prices  without  additional  benefit, 
they  are  no  longer  fair  and  reasonable,  and  the 
Government  is  not  bound  by  its  terms. 

B.  Types  of  Markets.  Government 
contracting  organizations  essentially  operate  in 
three  types  of  markets: 

1.  the  major  systems  environment. 
This  is  highly  structured  on  both  the  Industry 
and  Government  side,  with  specific  business 
units  established  to  support  Government 
customers  and  special  rules  (e.g.,  cost 
accounting  standards)  which  apply  only  to 
them.  Due  to  the  unique  nature  and  use  of 
these  products,  commercial  market  forces  and 
comparisons  are  irrelevant.  Military  or 
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Government  unique  information  systems  are 
included  here. 

2.  general  purpose  commercial 
products.  The  Government,  using  small 
purchase  procedures,  the  Federal  Supply 
Schedule  or  IFB  procedures  for  large  quantity 
buys  of  commercial  and  stable  products  and 
pricing,  generally  performs  well. 

3.  commercial  information  systems. 
While  not  generally  recognized  as  a  separate 
market,  it  should  be  distinguished  from  the  first 
two  types  due  to  the  evolutionary  and 
revolutionary  ongoing  changes  in  price  and 
technology,  and  to  the  large  dollar  volume  of 
Federal  purchases  involved.  While  the  Federal 
Government  does  not  dominate  this  market, 
and  may  represent  less  than  10  per  cent  of  it 
overall,  individual  contracts  in  this  market  are 
expected  to  exceed  $50  million  (and  often 
more);  so  certain  economies  would  be 
expected  to  accrue  to  the  Government 
customer  and  also  to  provide  other  market 
influence/leverage.  This  suggestion  only 
discusses  this  market,  but  as  these  products 
may  be  acquired  by  the  Government  here  and 
GFE'd  (provided  as  Government  furnished 
equipment)  to  systems  contracts  to  prime 
contractors  when  the  Government  is  the  end 
user,  it  addresses  this  potential  use  as  well. 

C.  Current  Process  Shortcomings 
Identified  by  National  Leaders.  Government 
acquisition  of  information  products  are  not  as 
effective  as  they  could  be,  as  discussed  in  Mr 
Kelman's  "Procurement  and  Public 
Management  (Atch  2):" 


1.  Due  to  falling  prices  and  rapid 
technologic  change/improvement,  prices 
successfully  bid  in  response  to  RFPs  are 
largely  irrelevant  (p.  30)  given  either: 

a)  products  other  than  those  in 
the  original  award,  which  are  noncompetitively 
added  after  award  under  technology 
refreshment  clauses,  and  priced  in  a 
noncompetitive  environment  (the  National 
Performance  Review  complains  of  hopefully 
occasional  price  gouging),  predominate  what  is 
actually  delivered,  or 

b)  products  that  were  state  of 
the  art  (or  at  least  current)  when 
awarded/negotiated,  are  obsolete  when 
delivered  (according  to  the  National 
Performance  Review),  or 

c)  even  without  product 
change,  prices  for  those  products  in  the 
successful  proposal  often  drop  sharply  in  the 
commercial  market  place.  Accordingly, 
savings  obtained  reflect  the  time  taken  to  make 
an  award,  as  well  as  competition.  A  prompt 
award  represents  a  financial  risk  to  the 
awardee,  who  might  seek  procedural  or  actual 
delivery  delays  to  get  well;  a  situation  which 
may  have  occurred.  This  may  be  why  the 
National  Performance  Review  cites  a  "286 
computer  offered  the  Government  at  a  486 
price." 

This  questions  the  value  obtained  by  the 
competition  process  for  these  products. 
Certainly,  the  competitive  process  costs  time 
and  money,  for  both  Government  and  Industry; 
so  there  should  be  a  corresponding  benefit.  To 
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the  extent  there  is  not,  a  new  process  is 
required  to  obtain  it. 

2.  Due  to  the  difficulty  of 
forecasting  future  requirements  and  available 
funding,  as  well  as  market  changes  that  are  not 
forecast,  product  mix  and  deliveries  reflected 
for  proposal  evaluation  may  not  be  reflective  of 
that  actually  ordered  (pp.  30-1).  While  this 
may  not  be  possible  to  overcome  as  a 
shortcoming,  and  represents  a  methodology 
that  applies  to  all  offerors  so  that  there  is  no 
competitive  advantage  to  any  firm,  it  is 
nonetheless  an  issue  which  calls  into  question 
the  merit  of  evaluating  a  specific  proposed 
product  solution  as  a  basis  for  award.  Instead, 
such  other  as  factors  as  realistically  evaluated 
past  performance,  in  terms  of  supporting 
customers  and  meeting  crises,  could  therefore 
be  much  more  valuable. 

3.  Past  performance,  as  a  source 
selection  criterion,  may  not  be  as  effective  as  it 
could  be  due  to  the  nature  of  the  current 
process  (pp.  41-53).  Personal  history  can  not 
be  factored  in  and,  typically,  the  offeror  is 
asked  to  provide  a  listing  of  references,  and  can 
be  expected  to  make  a  self  serving  selection. 
Also,  performance  reports  are  by  their  nature 
subjective,  and  present  other  problems,  such  as 
their  currency  and  relevancy,  that  may  not  be 
sustained  if  the  source  selection  is  sensitive  to 
it.  As  a  result,  scores  for  this  factor  are 
generally  the  same.  Mr  Kelman,  from  a 
common  sense  perspective,  argues  this  is 
perhaps  the  most  valuable  criterion  and  is  used 
more  extensively  in  industry;  and  the 
Government  not  using  it  is  a  major  failure  of 
the  current  process. 


4.  There  is  little  effective  control  over 
the  contractor,  given  the  prospect  of  a  two  year 
period  as  a  minimum  to  award  a  new  contract 
(P  54). 

5.  Industry's  approach  to  source 
selection  is  more  flexible,  offering  the 
opportunity  to  obtain  and  utilize  information  up 
to  award.  Further,  it  continues  into  the 
performance  period,  where  users  leam  more 
about  what  they  want  from  what  they  use  — 
how  features  can  become  utilized  as  previously 
unnoticed  benefits  (pp.  57-8).  In  fact, 
Government  acquisitions  are  often  grand 
design  projects  that  lay  out  too  many  details  in 
advance  (p.  88),  particularly  in  a  world  and  in  a 
market  place  that  changes  greatly  and 
frequently  (p.  89). 

6.  Long  term  business  relationships  are 
difficult  to  establish,  given  the  Government's 
inability  to  effectively  utilize  or  recognize  past 
performance.  As  a  result,  good  ideas  that 
could  be  offered  to  the  Government  during 
contract  performance  may  be  withheld,  so  they 
can  be  used  by  that  firm  in  future  competitions. 

If  offered  earlier,  the  contractor  would  not  be 

rewarded  for  his  ingenuity,  but  that  information 
would  be  generally  released  to  interested  US 
persons  (pp.  64-6,  74).  In  Industry,  little 
documentation  is  required  or  delays 
encountered,  and  high  quality  performance 
provided,  simply  because  vendors  know  their 
customer  will  tie  future  awards  to  current 
performance  (p.  68). 

7.  Site  and  program  specific 
knowledge  can  be  developed,  with  benefit  to 
both  parties,  over  a  long  term  relationship;  and 
additional  benefit  from  informal  exchange  of 
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information  between  the  parties'  personnel  and 
from  capital  investments  made  when  justified 
by  those  long  term  relationships  (pp.  69-70). 

8.  Public  officials  do  not  have  the 
discretion  to  exercise  judgement  and  common 
sense  in  the  sense  of  what  information  systems 
to  buy  or  how  to  use  them.  By  this,  DOD 
should  consider  the  military  user's  need  for 
flexibility  and  good  judgement,  rather  than  a 
contracting  process  that  does  not  provide  it  (p. 
90)  (with  comparable  considerations  by  other 
Departments  for  their  users). 

HI.  PRINCIPLES.  These  correlate  with 
Vice  President  Gore's  National  Performance 
Review. 

A.  Cut  Red  Tape.  Program  Management, 
contract  management,  legal  and  quality 
assurance  time,  effort  and  personnel,  as  well  as 
now  unneeded  documents,  would  be 
substantially  reduced  through  use  of  electronic 
catalogues,  ordering  and  payment. 

B.  Put  Customers  First.  Prime  contractors, 
once  selected,  will  be  expected  to  support 
customers  professionally  in  every  sense.  If  not, 
a  mechanism  would  be  in  place  to  replace  one 
or  more  of  them.  In  the  interim,  orders  would 
be  written  against  the  other,  existing  MARC 
primes,  continuing  effective  competition.  In 
this  manner,  poor  firms  or  ones  that  become  so 
would  be  effectively  excluded  from  future 
work,  which  would  be  diverted  to  current  and 
future  MARC  primes;  and  contractual 
provisions  would  still  require  them  to  fix 
already  delivered  products. 


C.  Empower  Employees.  Users  will  be  able 
to  freely  select  from  the  catalogues  MARC 
primes  present  without  unnecessary 
interference  or  delay  from  administrative 
personnel.  Similarly,  the  MARC  primes  will  be 
empowered  to  offer  the  most  attractive 
♦products  and  prices  without  administrative 
delay,  such  as  for  the  negotiations  and 
modification  preparation  time  now  required  by 
technology  refreshment  clauses.  In  military 
terms,  we  expect  our  warriors  to  achieve  their 
objectives  with  the  available  resources.  We 
have  the  good  fortune  to  possess  warriors  with 
the  attributes  (flexibility,  independence,  good 
judgement  (and  so  forth)  to  prevail  in  war. 
Military  users  selecting  from  competing 
MARC  catalogues  would  be  empowered  to 
select  the  most  competitive  products  and  then 
prevail  in  a  business  sense  as  well. 

D.  Go  Back  To  Basics.  With  the  flexibility, 
ease  of  use  and  accountability  thus  obtained, 
from  the  standpoint  of  both  the  Industry  and 
Government  partners,  the  trust  deficit  would  be 
closed  and  efficiencies  obtained.  Users  would 
clearly  see  results,  and  adjust  accordingly 
without  intermediaries  confusing  issues,  as 
would  MARC  contractor  senior  managers  to 
the  extent  improvement  is  required  from  their 
standpoint.  Gamesmanship  would  be 
eliminated,  with  any  "special"  or  "frill" 
requirements  from  non-user  (third  party) 
Government  sources  accommodated,  but 
priced  separately.  Actions  taken  would  not  be 
"by  the  book,"  but  only  what  makes  sense  to 
users  who  would  then  have  the  flexibility  to 
make  the  decisions  they  consider  appropriate; 
and  whose  superiors  could  then  solely  hold 
them  accountable  for  the  outcomes.  Due  to 
these  improvements  and  those  in  the  "Cut  Red 
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Tape"  paragraph,  internal  Government 
organizations  could  also  be  simplified. 

IV.  APPROACH: 

"Multiple  Acquisition  Requirements  Contracts" 
(MARCs),  are  called  "requirements  contracts" 
because  user  requirements  within  each  sector 
must  normally  be  satisfied  against  one  or  more 
of  the  set  of  MARC  contracts,  similar  to  the 
current  case  if  a  single  requirements  contract 
was  issued.  They  are  termed  "multiple" 
because  this  approach  is  modified  by  a 
Government  right  to  go  to  any  or  all  of  the 
prime  contracts  in  the  MARC  set.  If  a  user 
determined  no  firm  in  their  MARC  set  could 
match  or  improve  on  those  other  available 
methods,  they  would  not  be  bound  to  do 
something  dumb  (as  currently,  another  source 
could  be  justified  because  the  price  is  not  fair 
and  reasonable)  but  could  accept  the  better 
offer.  Also,  they  would  briefly  document  this 
situation,  and  send  a  copy  to  the  MARC  set's 
Government  contracting  officer  for 
management  action. 

The  overall  information  systems  market  could 
be  divided  into  sectors,  reflecting  different  user 
types  (e.g.,  office  automation,  workstations, 
conference  room/auditoria).  If  deemed  to  be 
unwieldy,  a  different  division  might  be 
preferable  (by  Agency,  Command  or  by 
geographic  area).  Rather  than  these  contracts 
being  a  grand  design,  all  things  to  all  users,  and 
possibly  not  being  truly  responsive  to  anyone, 
another  approach  would  be  to  define  discrete 
user  groups  so  the  MARC  prime  contractors 
would  be  more  focused.  Of  course,  the  cost 
(e.g.,  administrative  and  lower  business 
volume)  would  be  compared  with  the  benefit 


(e.g.,  more  focused  approach,  more  tailored 
products);  and  the  size  or  shape  of  the  user 
group  determined  accordingly. 

A.  Terminology.  The  suggested  approach 
envisions  an  initial  competition  to  establish 
groupings  (or  "sets")  of  two  or  more  (e.g., 
three)  prime  contractors;  each  MARC  set 
serving  a  unique  user  type,  (or  "user  group)." 
There  would  be  one  MARC  set  supporting 
each  market  sector. 

B.  General.  Each  set  will  compete  for  work 
on  the  basis  of  best  value,  and  will  be  able  to 
communicate  with  users  without  artificial 
barriers,  and  utilize  electronic  tools  to  do  so. 
The  users  are  empowered  to  go  to  whichever 
firm  they  chose  to  in  their  set,  utilize  other 
firms  in  the  set  as  circumstances  warrant, 
optimize  best  value  offerred  and  be  accountible 
for  results  more  than  ever  before.  Similarly,  the 
prime  contractors  comprising  each  set  would 
freely  choose  the  range  of  vendors,  products 
and  services  —  and  the  prices  —  they  would  like 
to  offer  the  users;  duly  noting  the  need  to  be 
competitive. 

1.  Awards.  Three  (or  so,  depending 
on  volume  of  business  expected,  the  nature  of 
the  vendor  base  and  the  characteristics  of  the 
user  group)  prime  contracts  would  be 
competitively  awarded  to  support  identified 
users.  Either  no  or  a  small  volume  of  business 
would  be  associated  with  the  award.  Each 
prime  contractor  would  publish  a  priced  list  of 
goods  and  services  (see  "updates")  from  which 
orders  could  be  placed  by  authorized  users 
without  further  competition  (similar  to 
Indefinite  Delivery  contracts). 
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Consideration  to  the  prime  contractors  in  the 
set  would  be  the  Government's  promise  to  fill 
requirements  from  the  user  group  by  orders 
issued  to  one  or  more  of  those  firms. 
Consideration  to  the  Government  would  be  the 
expectation  those  firms  would  offer  better 
business  deals  through  this  process  than 
otherwise  available. 

2.  Catalogues.  The  prime  contractors 
within  each  set  would  periodically  and 
unilaterally  establish  listings  ("catalogues")  of 
products  and  prices  from  which  the  user 
selection  is  freely  made.  This  would  be  done 
electronically,  such  as  by  bulletin  board  or  by  a 
data  base,  minimizing  administrative  effort  and 
delay  (also,  see  "updates");  and  also  tie  to 
ordering  and  payment  functions.  The  users 
could  select  products  or  services  from  any  or 
all  of  the  current  catalogues  offered. 

3.  Updates.  Periodically,  perhaps 
bimonthly,  and  simultaneously,  each  prime 
contractor  would  update  their  catalogue,  in 
new  "offering  periods;"  thereby  incorporating 
new  or  updated  products  and  pricing  as  they 
would  like  without  negotiations. 

The  changes  would  all  be  published  on  the 
same  date,  and  be  effective  on  the  same 
(possibly  a  different)  date.  The  updates  would 
periodically  occur,  with  the  timing  dependent 
on  the  conditions  in  each  given  sector, 
balancing  beneficial  extent  of  the 
price/technology  change  in  the  marketplace 
with  the  user  sector's  cost  and  difficulties 
associated  with  implementing  each  change. 

4.  Scope.  Product  and  services 
updates  must  be  appropriate  for  the  broadly 


defined  requirements  of  the  users.  For 
example,  mainframe  computers  might  not  be 
appropriate  for  office  automation  users,  but 
new  generations  of  personal  computers  and 
normally  associated  products,  while  new, 
would  be  --  while  the  scope  would  now  really 
be  defined  as  and  by  the  market  sector.  To 
restate  the  point,  the  scope  might  be  the  current 
commercial  state  of  the  art  for  office 
automation,  and  it  must  be  noted  that,  over 
time,  leading  or  esoteric  technology  would  be 
introduced  into  the  commercial  market  place, 
and  therefore  enter  within  the  contract  scope. 
However,  these  products  must  always  be 
commercially  available. 

Negotiations  would  not  be  required  in 
conjunction  with  these  changes,  due  to  the 
commercial  nature  of  the  products,  the 
responsibility  expected  of  the  MARC  prime 
contractors  and  the  essential  driving  nature  of 
competition  in  this  environment,  rather  than  its 
appearance.  Instead  of  relying  on  negotiations, 
the  linkage  with  the  marketplace  would  ensure 
value  to  users.  If  noncompetitive  primes 
existed,  they  would  not  likely  get  business  until 
the  next  offering  period,  at  which  time  they 
could  offer  a  more  competitive  product/price 
listing,  as  would  their  competitors.  That  firm's 
need  to  recover  their  management  cost  would 
be  unsatisfied,  due  to  the  low  ordering  rate  — 
and  the  prospect  of  a  loss  contract  would 
mitigate  against  their  initially  competing  or  to 
their  willing  removal  when  faced  with  more 
competitive  firms  (or  to  their  unwilling  removal 
or  lapse  under  the  contract's  terms). 

5.  Competition.  The  basis  for 
selection  is  discussed  in  the  "source  selection" 
paragraph.  Once  the  prime  contracts  were  in 
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place,  they  would  essentially  be  competing  for 
work  against  one  another  by  offering  best  value 
(better  products,  delivery  or  price,  and  by  their 
business  relationships  with  users).  Users  would 
migrate  to  the  best  firms,  and  away  from  those 
they  were  dissatisfied  with,  by  simply  placing 
orders,  without  any  requirement  for  justifying 
their  doing  so.  Administrative  procedures 
would  be  in  place,  as  currently,  if  contractual 
remedies  are  required  on  ordered  items.  Once 
awarded,  given  competition  would  be 
essentially  continuous,  the  MARC  contracts 
need  not  have  a  fixed  termination  date,  but  new 
prime  contractors  would  be  added  as  needed 
(e.g.,  to  replace  poor  ones,  to  provide  added 
capacity,  to  exert  additional  competitive 
pressure). 

The  commercial  marketplace  offers  potentially 
higher  profit  margins  than  Government 
negotiators  would  allow,  but  cost  and  pricing 
data  has  always  been  difficult  to  obtain  or 
verify.  The  Government  need  not  obtain  this 
data  if  it  can  obtain  the  benefit  of  it,  lower  price 
and  superior  products,  by  the  use  of  this 
technique.  Again,  that  benefit  may  not  be 
obtained  if,  for  example,  mega  buy/IDIQ 
contracts  could  not  obtain  prices  that  were 
significantly  lower  than  that  available 
commercially  or  on  GSA  schedules. 

6.  Source  selection.  I  will  discuss  why 
source  selection  should  not  be  made  only  on 
the  basis  of  a  proposed  solution,  and  suggest  a 
new  approach  to  source  selection  criteria. 

Typically,  sources  are  currently  selected  on  the 
basis  of  proposed  solutions  to  the  situation 
described  in  the  solicitation.  Due  to  the 
difficulty  of  forecasting  actual  conditions  (as 


described  in  Kelman's  book),  and  to  the 
certainty  of  radical  change  in  the  information 
products  marketplace,  it  is  problematic  whether 
this  approach  is  practical,  relevant  or  obtains 
the  best  result.  However,  it  is  clear  that,  while 
circumstances  change,  the  ability  to  provide 
customer  support  and  to  engender  positive 
business  relationships  are  desirable  —  and  a 
firm's  process  of  doing  so,  and  results  obtained, 
may  be  more  relevant  than  a  forecast  approach 
to  a  hypothetical  scenario. 

In  the  current  political  and  economic 
environment,  the  ability  to  identify  and  utilize 
information  systems  and  products  in 
imaginative  as  well  as  productive  ways  and  to 
cope  with  change  is  critical,  and  can  be 
documented.  Offerors  should  describe  how 
well  they  have  done  so. 

A  source  selection  factor  for  proposed  business 
mechanisms  to  be  used  to  facilitate  instant 
contract  and  program  management,  to  facilitate 
user  decision  making  or  to  manage  orders 
during  contract  performance,  such  as  use  of 
Electronic  Data  Interchange,  should  be 
considered. 

Offerors  that  act  as  integrators  would  be 
expected  to  have  broad  market  knowledge  of 
current  and  future  trends,  technologies  and 
products;  and  expend  significant  costs  for  that 
capability,  supporting  both  current  customers 
and  that  firm's  competitiveness  for  future  work. 
Proposals  could  therefore  allow,  but  not 
require,  offerors  to  share  that  knowledge  with 
selected  Government  managers.  The 
information  would  be  used  for  planning 
purposes,  respecting  the  MARC  contractor's 
proprietary  interest,  and  Government 
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personnel,  as  with  other  proprietary 
information,  would  be  liable  for  prosecution  if 
they  failed  to  protect  it  (as  currently);  or 
transferred  it  to  other  firms  (e.g.,  technical 
leveling).  Only  the  contractor's  presentation 
costs  would  be  recognized  by  the  Government, 
because  that  information  and  the  cost  of 
collecting  it  is  already  incurred  for  their 
business  purposes,  but  award  fee  may  also  be 
paid  when  significantly  valuable. 

7.  Administration.  No  negotiations 
and  less  bid  and  proposal  effort  (due  to  the 
long  term  nature  of  the  contracts)  would  be 
required,  with  concommitant  savings  for  both 
parties,  which  due  to  competition  would  be 
reflected  in  lower  contract  prices. 

The  Government  would  not  be  locked  in  if  no 
MARC  prime  contractor  offerred  fair  and 
reasonable  prices.  In  that  case,  a  simple 
justification  would  document  purchases  not 
placed  under  MARC  procedures,  and  would  be 
required  due  to  the  Government's  consideration 
originally.  In  the  unlikely  event  this  occurs,  a 
copy  of  the  filed  document  would  be  sent  by 
the  ordering  organization  to  the  MARC 
contracting  officer  for  review  and  possible 
management  action. 

8.  Quality  Assurance  (QA).  Due  to 
the  high  reliability  of  commercial  information 
products  today,  testing  and  QA  should  not  be 
required.  Each  MARC  prime  contractor  and 
its  lower  tier  suppliers  will  be  responsible  for 
satisfying  customers,  and  more  extensive 
Government  quality  assurance  would  be 
unwarranted,  unless  desired  by  a  user  on  a  fee 
for  service  basis.  Each  prime  would  be 
responsible  for  any  contractual  warranties 


desired  by  the  users,  and  vendors'(suppliers  to 
primes)  standard  commercial  warranties  in 
excess  of  contractual  requirements  would  flow 
to  the  Government.  As  stated  elsewhere, 
continuing  user  problems  with  any  prime  would 
result  in  present  day  type  contractual  action  on 
existing  orders  as  well  as  naturally  shifting 
future  orders  away  from  poor  primes  to  the 
ones  that  satisfy  users  best.  If  extended 
warranties  are  desired  by  users,  they  would  be 
made  available/required  and  an  additional 
charge  could  be  established  for  them. 

9.  Contract  structure.  Separate 
contract  line  items  would  be  established  for 
baseline  items  and  for  additional  support.  For 
example,  computers  could  be  ordered  on  a  no 
frills  basis  (drop  ship  from  manufacturer  to  be 
installed  by  Government  or  local  contract 
support  personnel),  minimal  service  (in 
metropolitan  areas,  use  of  a  credit  card  for 
walk-in  service  at  a  firm's  warehouse  or  their 
vendor's)  or  full  service  basis  (the  prime  installs 
products  at  the  Government  site). 
Alternatively,  rather  than  contracts  which  serve 
the  full  range  of  customers,  separate  contracts 
only  for  full  service  MPKs  and  others  only  for 
limited  service  MPKs  could  be  issued  if 
deemed  more  workable.  As  stated  elsewhere, 
associated  upgrades,  warranties  or  related 
services  could  be  separately  priced.  Also, 
multiple  product  groupings  (sometimes  called 
configurations)  for  a  standard  work  unit  could 
be  offered,  which  typically  present  savings. 

10.  Program  Management. 
Government  employee  use  could  be  minimized, 
with  only  direct  and  supervisory  employees 
required  for  contract  operations.  The  approach 
would  totally  be  on  a  value  added,  fee  for 
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service  basis,  and  not  subject  to  normal 
administrative  controls. 

Administrative  effort  could  be  totally  paid  for 
by  the  requestor  (e.g.,  Freedom  of  Information 
Act  requests  would  be  satisfied  by  program 
support  employees  whose  salaries  would  be 
totally  paid  by  associated  charges,  and  perhaps 
contracted  out  to  an  associated  administrative 
arm  as  now  offerred  by  the  US  Department  of 
Commerce's  National  Technical  Information 
Service).  Similarly,  users  or  other 
organizations  such  as  their  headquarters 
requiring  reports  would  be  expected  to  directly 
pay  for  them,  with  appropriate  accounts 
established  in  advance  for  them.  This  could  be 
difficult  for  the  Government  to  accept,  but 
visibility  of  operational  costs  and  accountability 
for  other  ("overhead"  and  non  value  added) 
effort  would  be  required  if  we  would 
demonstrate  the  value  of  this  practice  and 
expect  competitive  practices. 

An  amount  would  be  added  to  each  order, 
perhaps  as  a  percentage  of  the  order  value,  to 
fond  the  employees's  salaries  and  overhead 
costs.  An  additional  amount  to  be  levied, 
without  interchange  with  the  first  pool,  to 
incentivize  the  primes  to  be  determined  and 
described  by  the  users. 

User  group  boundaries  must  be  considered 
carefully.  Small  Agencies  or  other,  larger 
Agencies  with  low  volumes  of  orders  at  widely 
distributed  sites  may  not  warrant  inclusion  to 
the  extent  they  would  be  detrimental.  This 
would  be  the  case  if  the  administrative  burden 
of  carrying  them  would  significantly  reduce  the 


business  deal  that  would  be  offered  by  primes 
otherwise  to  large  volume,  well  managed  users. 

The  MARC  contracts  are  in  effect  a  tool,  not  a 
program.  Accordingly,  no  separate  program 
management  function  would  be  required. 
Controls  would  be  as  currently  established  for 
funding  or  information  systems  oversight.  The 
contracting  organization  would  satisfy  any 
additional  oversight  functions  required.  Each 
sector  would  be  separately  managed  by  the 
contracting  organization  responsible  for  it,  with 
periodic  user  meetings  and  subsequent 
Govemment/prime  contractor  meetings  as 
required.  Use  of  MARC  would  satisfy 
competition  requirements,  and  the  volume  of 
purchases  expected  should  save  users  time  and 
resources  accordingly. 

Government  information  that  is  currently 
acquisition  sensitive  could  now  be  shared 
(between  the  Government  and  each  contractor, 
but  not  between  contractors),  given  the  MARC 
contract's  long  term  nature  reduces  the  need  for 
annual  or  frequent  competitions  and  the  need 
for  categorization  of  that  information  as  such. 
In  turn,  sharing  information  freely  has  a 
beneficial  effect,  and  fosters  business 
relationships  that  become  more  of  a 
partnership.  For  example,  we  may  expect  that, 
with  shared  information,  planning  and  therefore 
outcomes  will  improve. 

1 1 .  Comparison  with  other  techniques. 
Indefinite  delivery  contracts,  as  described  in  Dr 
Kelman's  book,  are  not  sufficiently  flexible, 
have  high  overhead  costs,  and  do  not  offer  the 
best  value  to  customers. 
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Bulletin  boards  have  been  successfully  used  to 
overcome  the  problems  associated  with  the 
prior  contract  type,  but  require  periodic  listings 
of  new  requirements  and  are  more  manpower 
intensive  than  the  MARC  approach;  offering 
administrative  cost  savings  in  addition  to 
heightened  customer  satisfaction.  Also,  due  to 
the  high  volume  of  work,  economies  of  scale 
would  be  experienced  versus  the  short  term 
and  limited  value  of  the  many  small 
competitions  that  characterize  the  bulletin 
board  approach. 

GSA  schedules.  Currently,  these  schedules  are 
awarded  for  predetermined  periods  of  time, 
with  set  products  and  pricing,  and  with  lower 
levels  of  work.  Also,  they  require  a  significant 
effort  on  the  part  of  Industry  and  the 
Government  to  negotiate  and  to  maintain. 
Accordingly,  the  MARC  approach  is  more 
flexible,  more  likely  to  promptly  recognize 
price  reductions  and  product  improvements, 
and  to  have  a  lower  administrative  cost  for 
both  parties. 


12.  Standards.  Government  standards 
have  been  sought  and  often  published,  with 
mixed  results.  Well  intended  standards  offer 
theoretic  commonality  and  connectivity 
advantages;  but  it  has  been  argued  that  they 
also  have  a  higher  initial  cost,  users  are 
unwilling  to  pay  for  the  higher  cost  they  bear  in 
practice  and  their  intent  is  now  achieved  in 
open  systems  as  driven  by  competitive 
pressure. 

The  essential  flexibility  of  the  MARC  approach 
will  afford  the  user  access  to  market  driven 
forces  and  results  —  including  the  flexibility  to 
be  aware  of  and  react  to  products  that  become 
more  or  less  attractive  due  the  interaction  of 
evolving  standards  in  a  commercial 
marketplace  —  and  then  the  user  will  be 
responsible  for  the  decision  made  in  accordance 
with  any  direction  imposed  by  his  management 
process. 
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DEFINITIZING  CHANGE  ORDERS 
REVOLUTIONARY  THINKING 
JAMES  H.  GILL 


ABSTRACT 

Conventional  wisdom  with  regard  to 
change  order  definitization  states 
that  it  is  in  the  government's 
interest  to  negotiate  and  definitize 
change  orders  at  the  earliest 
possible  time.  On  incentive 
contracts,  traditional  thought 
teaches  us  that  the  contractor  will 
work  efficiently  to  minimize  cost 
incurrence  thereby  saving  money  for 
the  government  (cm  the  share  line) . 

While  this  is  often  true  when  dealing 
with  responsible  contractors,  it  is 
predicated  upon  an  assumption  of  a 
negotiated  fair  and  reasonable  price. 
I  will  postulate  that  it  can  be 
demonstrated  that  there  are 
contractors  for  whan  this  behavior  is 
tantamount  to  "giving  away  the  farm". 
There  are  sane  contractors  for  whan, 
for  a  variety  of  reasons,  it  is  more 
beneficial  to  the  government  to  delay 
negotiations  until  most,  if  not  all, 
of  the  effort  is  complete.  The 
resultant  actuals  can  translate  into 
substantial  savings  to  the  government 
when  compared  to  negotiated 
settlements  which  comply  with  the  DoD 
recommended  schedule  (180  days  or 
less) . 

This  article  will  provide  factual 
data  which  will  demonstrate 
conclusively  the  merit  in  negotiating 
actuals  in  certain  instances.  While 
limiting  the  scope  of  the 
recanmendation  by  the  nature  of  the 
acquisition  and  the  performance  of 
the  contractor,  a  general  exception 
to  the  traditional  change  order 
definitization  process  is 
appropriate. 

I  will  analyze  the  results  of  data  on 


change  activity  with  two  contractors 
fitting  the  appropriate  profile  to 
shew  the  savings  realized  through 
this  technique.  While  I  recognize 
that  this  approach  is  subject  to 
significant  criticism,  it  reflects 
the  real  world  and  the  associated 
difficulties  in  gcvernment/oontractor 
relations. 

D7IROOJCTTCN 

Uhdefinitized  Contract  Actions  or 
UCA's  exist  because  it  is  an 
inperfect  world  and  there  is  not 
always  tune  to  do  things  the  "right" 
way.  Conditions  often  conspire  to 
force  award  of  a  contract  or  change 
order  in  an  abbreviated  time  period 
due  to  schedule  constraints.  Under 
these  circumstances  conventional 
wisdcra  would  have  us  maximize  effort 
to  definitize  these  efforts  quickly 
to  avoid  the  "blank  check"  syndrome. 
Once  authorization  to  proceed  with  an 
effort  has  been  provided  to  a 
contractor,  he  must  be  incentivized 
to  control  costs  in  order  to  preclude 
him  from  running  up  a  large  bill  of 
"actuals"  and  presenting  them  to  the 
government  for  payment  and  inclusion 
into  the  contract  target.  These 
actuals  are  extremely  difficult  (if 
not  impossible)  to  negotiate  away. 


Traditional  thinking  would  have  us 
believe  that  risk  is  mitigated  by 
early  negotiation  of  a  "target"  cost 
against  which  the  contractor  will 
more  efficiently  manage  the  effort  if 
he  is  to  maximize  profit.  Incentive 
contracting  normally  has  at  its 
essence  the  principle  that 
contractors  are  motivated  primarily 
by  profit  and  will  not  incur 
unnecessary  costs  which  they  would 
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then  have  to  share  with  the 
government  under  the  incentive 
arrangement. 

While  this  is  normally  true,  there 
are  exceptions  such  as  the 
maintenance  of  a  workforce  during  lag 
periods  when  a  contractor  is  between 
programs .  During  the  current 
downsizing  of  defense  related 
industries  this  has  become  more  of  a 
problem  than  in  prior  years  when 
contractors  were  expanding  to  meet 
the  increases  in  the  Defense  budgets. 

We  will  accept  the  premise  that 
contractors  normally  are  motivated 
primarily  by  profit  and  will  seek  to 
maximize  it.  Is  it  not  then  sensible 
to  negotiate  a  target  cost  as  early 
as  possible  in  order  to  put  the  onus 
on  the  contractor  to  minimize  costs? 

The  answer  to  this  question  is  the 
traditional  Defense  Systems 
Management  College  (DSMC)  answer,  it 
depends.  It  depends  upon  the 
government's  ability  to  ensure  that 
the  target  cost  is  fair  and 
reasonable.  If  the  negotiated  target 
cost  is  unreasonably  high,  then  the 
fact  that  the  target  cost  was 
negotiated  in  a  timely  manner  is  of 
minimal  relevance,  as  is  the 
contractor ' s  eventual  success  in 
underrunning  the  target. 

Change  Orders  by  definition  involve 
effort  that  is  time  sensitive,  i.e. 
if  the  effort  was  not  critical ,  a 
supplemental  agreement  would  be 
accomplished.  Proposals  for  this 
effort  normally  rely  heavily  upon 
engineering  estimates  rather  than 
actual  history.  Contractors  normally 
inflate  these  estimates  toward  the 
high  end  (estimates  for  developmental 
work  will  normally  have  a  broad  range 
of  variability) . 

Most  contractors  will  have  some 
negotiation  flexibility  within  their 
initial  proposal.  Seme  contractors 
are  more  flexible  than  others  with 


regard  to  their  willingness  to  move 
from  their  initial  proposal.  In  a 
perfect  world,  both  parties  would 
reach  an  agreement  on  a  target  cost 
that  reflected  a  shared  risk  that  the 
contractor  would  realize  a  fair 
profit  if  they  perform  the  effort 
competently. 

Unfortunately,  there  are  extraneous 
factors  that  complicate  the  process. 
One  of  these  factors  is  the  contract 
type  (i.e.  fixed  price  incentive)  and 
its  relationship  to  a  ceiling  price. 
If  a  change  order  is  issued  to  a 
contract  in  which  the  contractor  is 
facing  an  over  ceiling  position,  i.e. 
one  in  which  he  is  assuming  a  share 
ratio  of  0/100%,  it  is  more  than 
likely  that  he  will  attempt  to  use 
the  change  order  to  recover  seme  of 
the  ceiling  dollars .  In  this 
environment,  the  contractor  is  less 
interested  in  profit  than  in  ceiling 
price.  The  ceiling  price  is  a  direct 
factor  of  target  cost,  therefore,  the 
contractor  will  attempt  to  maximize 
the  target  cost  adjustment  for  the 
change  order.  In  this  environment  it 
is  difficult  for  the  government  to 
convince  the  contractor  that  his 
proposed  cost  is  "unreasonably"  high. 

Other  factors  may  also  come  into  play 
in  a  contractors  seeming 
intransigence  with  regard  to  target 
cost.  There  may  be  an  extremely 
adversarial  relationship  that  has 
resulted  from  previous  problems 
regarding  scope  of  contract.  Seme 
program  managers  assume  all  effort  is 
included  in  developmental  contracts 
(this  may  be  a  result  of  program 
budget  problems)  and  resists  issuing 
changes  to  the  contract.  Conversely, 
some  contractors  read  statement  of 
works  extremely  literally  and  attempt 
to  change  order  contracts  to  death. 
Developmental  contracts  normally 
require  some  ambiguity  in  their  SCW's 
due  to  the  nature  of  the  effort.  It 
is  this  ambiguity  that  makes  a  cost- 
plus  contract  a  more  reasonable 
vehicle  for  a  developmental  effort. 
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If  the  change  order  is  made  to  a 
cost-plus  contract,  the  risk  factor 
is  not  normally  the  inpetus  for  a 
contractor  to  hold  to  an  exorbitant 
estimated  cost.  On  a  cost  type  of 
contract  the  fixed  fee,  award  fee  or 
incentive  fee  is  a  factor  of  the 
estimated  cost. 

In  those  instances  in  which  a 
contractor  has  "bought  in"  (i.e. 
accepted  an  extremely  optimistic 
estimated  cost  for  the  basic 
contract) ,  the  motivation  may  be  to 
recoup  sane  additional  fee  that  was 
not  realized  with  the  award  of  the 
basic  contract.  Sane  contractors 
view  the  changes  clause  as  the 
perfect  vehicle  by  which  they  are 
able  to  get  well  -  a  Health  Plan  that 
even  Mrs  Clinton  would  admire. 

When  there  is  a  problem  with  the 
negotiation  of  a  fair  and  reasonable 
prioe  for  a  change  order,  it  is 
important  for  government  management 
to  use  discretion  in  the  constraints 
associated  with  the  time  guidelines 
imposed  upon  the  negotiation  team  for 
the  definitizaticn  of  contract 
actions.  In  certain  cases  it  may  be 
prudent  to  ail  lew  the  effort  to  go  to 
carpletion  before  it  is  definitized. 
This  obviously  will  not  be  a 
preferred  course  of  action,  however 
it  is  preferable  to  imposing  a 
unilateral  price  upon  the  contractor 
or  arbitrarily  throwing  money  at  the 
contractor  merely  for  settlement 
purposes. 

The  practice  of  negotiating  actuals 
is  certainly  not  the  policy  of  DoD 
contracting  offices,  however  there 
are  sane  benefits  to  be  realized  from 
this  course  of  action.  First,  it 
allows  the  actual  costs  to  be  used  as 
a  basis  for  profit.  Obviously,  this 
is  not  a  recommended  practice  in 
those  cases  where  a  contractor  has 
the  flexibility  to  run  up  enormous 
costs  outside  the  view  of  the 
government  representatives  (A00,  DFEO 
Quality  Inspectors,  DCAA,  Program 


Managers  etc.) . 

If  the  effort  is  relatively 
straightforward ,  and  the  government 
oversight  continues  to  be  effective, 
then  there  will  be  little  opportunity 
for  abuse  of  this  nature.  It  should 
not  be  necessary  to  wait  for  the 
entire  effort  to  be  completed,  merely 
enough  work  to  validate  the 
government's  technical  estimate. 
Only  in  rare  cases  would  actuals  to 
carpletion  be  required  to  reach 
agreement.  Second,  the  adjustment  to 
ceiling  price,  for  the  change  order 
effort,  will  be  minimized  if  the  risk 
has  been  reduced  by  incorporating 
actual  costs.  It  would  certainly  not 
be  necessary  to  give  the  contractor 
the  same  ceiling  price  rate  if  the 
risk  of  overrun  has  been  mitigated  by 
allowing  actuals  to  be  incurred. 

Third,  with  the  government  assuming 
much  of  the  risk,  the  profit  rate 
that  the  contractor  receives  will  be 
significantly  reduced. 

This  is  all  well  and  good  in  theory, 
tut  what  are  the  practical 
applications  of  this  in  the  reed 
world?  I  believe  that  it  can  be 
demonstrated  front  actual  cape 
histories  that  a  significant 
reduction  was  realized  through  the 
use  of  this  approach.  It  must  be 
stated  again,  that  this  is  an 
abnormal  situation  that  is  not  to  be 
confused  with  the  norm,  i.e. 
definitization  within  180  days  or 
less. 

BOOT 

Two  contractors  having  3  contracts 
with  the  government  were  selected 
that  met  the  parameters  identified 
above,  i.e.  they  were  over  ceiling  or 
they  had  a  history  of  intransigence 
in  negotiations  with  the  government. 
For  the  purpose  of  this  paper  it  is 
not  necessary  to  identify  these 
contractors  other  than  to  indicate 
the  rationale  for  including  them  as 
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candidates  for  use  of  actuals  in 
definitizing  change  orders. 

The  underlying  premise  behind  this 
research  is  to  assert  that  assessing 
timelines  to  definitization  of  change 
orders,  i.e.  180  days,  should  be 
tenpered  with  judgement  and  should 
therefore  be  situational.  If 
management  is  to  give  contracting 
officers  the  flexibility  to  decide 
appropriate  courses  of  action,  then 
this  flexibility  should  include 
disregarding  arbitrary  timelines 
whenever  it  is  deemed  appropriate. 

The  two  contractors  utilized  for  this 
study  fit  the  profile  in  two  ways. 
First,  they  are  historically  inclined 
toward  adversarial  behavior  in 
contract  negotiations.  They  tend  to 
assume  a  somewhat  arrogant  attitude 
in  that  they  presume  that  they  know 
what  is  best  for  the  customer. 

Further,  they  have  both  been  in 
negative  financial  situations,  both 
receiving  substantial  pressure  from 
their  corporate  headquarters  to 
improve  their  cash  flew  and  their 
financial  position  on  their 
respective  programs. 

The  first  contract  was  for  the 
development  of  a  guidance  system. 
The  contractor  was  initially  a 
relatively  small  R&D  company  who  had 
little  experience  in  the  actual 
production  of  hardware.  The 
government  encouraged  this  contractor 
to  compete  for  a  large  development 
contract  with  a  concurrent  production 
effort  to  follow.  The  contractor 
anticipated  a  large  return  on 
investment  to  be  realized  during  the 
follow-on  production  contracts. 

The  government  sought  to  broaden  the 
industrial  base  by  bringing  on  board 
a  potential  competitor  for  future 
guidance  work.  While  the  program  was 
of  a  fairly  large  dollar  value  ($300 
million)  the  organization  issuing  the 
contract  anticipated  several  other 


large  dollar  awards  in  the  future. 

This  contract  was  awarded  during  the 
build-up  of  the  defense  department 
during  the  Reagan  administration. 
Same  contractors  were  "buying  in" 
with  the  intent  to  recoup  profits 
during  subsequent  production  buys. 
Many  contractors  put  themselves  into 
extremely  bad  financial  conditions 
due  to  the  expectation  that  they 
could  not  lose  business  to 
competitors.  The  risk  was  compounded 
by  the  government's  advocacy  of 
fixed-price  research  and  development 
contracts,  many  of  which  were  awarded 
with  a  high  potential  for  going  over 
ceiling. 

Contractors  may  also  have  convinced 
themselves  that  they  merely  had  to 
break  even  in  order  to  place 
themselves  in  a  favorable  competitive 
advantage  for  future  business.  Some 
may  also  have  hoped  to  drive  out 
potential  competition  for  the 
anticipated  contracts  downstream. 

Early  in  the  initial  stages  of 
production  on  the  guidance  system,  it 
became  apparent  that  this  contractor 
did  not  realize  the  difficulties 
associated  with  the  start  up  of  a 
production  line.  It  was  also  evident 
that  the  contractor  did  not  have 
adequate  mid-level  managers  that  are 
critical  to  the  management  of  an 
effort  of  this  magnitude. 

The  government  program  office  had 
other  issues  that  would  subsequently 
impact  the  contractor.  The  Initial 
Operating  Capability  (IOC)  was 
critical  to  the  Service.  Premises 
had  been  made  to  the  Congress  which 
needed  to  be  kept.  The  Service  could 
not  field  the  system  without  this 
guidance  system.  The  program  team 
adopted  a  hands  on  approach  to 
solving  the  contractor's  problems. 

The  contractor  discovered,  subsequent 
to  contract  award,  that  the  defense 
buildup  would  not  happen  in 
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accordance  with  the  five  year  plan. 
Due  to  Congressional  direction,  a 
significantly  smaller  quantity  of 
guidance  systems  would  be  required. 
This  would  have  a  substantial  inpact 
upon  the  potential  profits  that  would 
be  available.  Further,  due  to  the 
disintegration  of  the  Soviet  Union, 
the  Service  would  be  seeing  a 
sizeable  reduction  in  its  funding  and 
a  good  portion  of  this  reduction 
would  impact  the  guidance  business. 

In  this  environment,  the  contractor 
looked  for  various  options  to  recover 
their  financial  position  on  the 
fixed-price  contract.  One  of  these 
options  was  to  file  claims  for  effort 
deemed  out  of  scope  on  the  production 
contract  and  shift  those  costs  to  a 
oost  plus  development  contract.  A 
second  approach  would  be  to  get  well 
through  changes,  ECP's  etc. 

The  government  contributed  to  the 
opportunities  available  to  the 
contractor  by  making  changes  which 
effectively  opened  the  door  to  the 
contractor  in  the  form  of  proposal 
opportunities.  Most  development  and 
production  contracts  have  numerous 
changes  to  them  during  the  course  of 
performance.  The  negotiation  of 
these  changes  proved  extremely 
difficult  as  the  contractor  appeared 
inflexible  and  somewhat  arbitrary  in 
his  justification  for  the  cost 
breakdown  of  the  requirements. 

During  this  period,  there  were 
numerous  changes  that  were  added  to 
the  contract.  The  government  was 
focusing  most  of  its  attention  on  the 
matter  of  recovering  schedule.  A 
good  amount  of  resources  were 
invested  in  assisting  the  contractor 
to  fulfill  his  contract  schedule  with 
a  quality  product.  The  contractor 
was  required  to  incorporate  several 
pieces  of  government  furnished 
property  in  the  performance  of  the 
contract.  The  contractor  alleged 
that  the  GFP  was  defective  or  late  in 
his  attempt.  to  recoup  both  cost  and 


schedule. 

During  the  factfinding  part  of  the 
negotiation  process  for  these  change 
orders  the  contractor  was  reticent  to 
provide  traceability  into  his 
proposal  methodology,  frequently 
changing  the  rationale  for  the 
inflated  number  of  hours  proposed. 
There  were  numerous  meetings  between 
the  government  team  and  the 
contractor  attempting  to  reconcile 
the  substantial  differences  between 
estimates. 

As  a  practical  matter,  some 
organizations  do  not  believe  in 
entering  into  negotiations  if  there 
is  a  difference  of  greater  than  10% 
between  the  government's  estimate  of 
the  number  if  hours  required  to 
perform  the  effort  and  that  of  the 
contractor's  updated  proposal.  The 
net  impact  of  this  approach  is  to 
prolong  the  factfinding  process. 

There  were  numerous  change  orders 
that  did  not  meet  this  standard  and 
were  referred  back  to  factfinding  in 
an  attenpt  to  get  the  sides  to  come 
closer  together.  Finally,  management 
interceded  on  several  instances, 
having  both  teams  present  their 
rationale  to  the  respective 
management  groups.  Management, 
having  escalated  their  involvement  in 
the  process,  would  then  discover  that 
the  problem  was  not  with  the 
respective  teams  or  individuals.  The 
problem  was  one  of  risk  assumption 
and  ceiling  adjustment. 

When  management's  involvement  was 
ultimately  unsuccessful,  a  brief 
sampling  of  the  efforts  that  had 
substantially  been  completed 
indicated  that  the  government  was 
realizing  a  sizeable  reduction  based 
upon  actual  costs  incurred.  The 
costs  actually  incurred  and  billed 
for  these  efforts  was  less  than  the 
government's  estimate  based  upon  the 
contractor '  s  proposal .  The 
government  ultimately  realized  a 


297 


lower  target  cost,  target  profit  and 
ceiling  price  than  they  would  have 
realized  if  the  contractor  had  merely 
accepted  the  government's  initial 
objective. 

There  may  be  several  reasons  for  this 
fact.  First,  both  parties  may  not 
have  understood  the  effort  well 
enough.  The  contractor  factored  in 
many  contingencies  that  might  not 
have  octae  true,  but  which  a  prudent 
risk  manager  could  not  afford  to 
overlook,  especially  in  an  fixed- 
price  environment.  Second,  and  most 
probable,  the  contractor  may  have 
been  atterrpting  to  recover  sore  of 
the  costs  that  had  fallen  into  the 
over-ceiling  amount.  It  is  not 
unreasonable  that  the  contractor  was 
anxious  to  get  back  to  a  position 
wherein  he  would  be  sharing  costs 
with  the  government.  Nothing  focuses 
senior  level  management's  attention 
more  quickly  than  the  need  to  use 
profit  to  pay  for  costs  incurred. 
The  quickest  way  for  a  contractor  to 
receive  a  bonus  is  to  move  costs  into 
the  sharing  arrangement  and  this  can 
be  accomplished  by  increasing  target 
cost  and  ceiling  price. 

Having  identified  the  problem,  what 
alternatives  were  available  to  the 
government?  The  contracting  officer 
could  issue  a  unilateral  modification 
which  would  be  subject  to  the 
"disputes”  procedures  and  cculd 
result  in  a  request  for  equitable 
adjustment  or  claim.  In  these 
instances  it  is  unlikely  that  this 
would  happen  because  as  a  rule  the 
ultimate  adjustment  for  costs 
incurred  would  be  less  than  the 
government's  objective  which  would  be 
the  basis  for  the  unilateral .  Claims 
can  create  a  significant  drain  can 
resources  as  they  require  a  good  deal 
of  work  in  processing  them  through 
the  Board/Ccurt. 

The  government  can  also  reduce  the 
ceiling  price  (and  profit  rate)  of 
the  eventual  supplemental  agreement 


due  to  the  reduced  risk  associated 

with  the  use  of  actuals.  With  this 
approach  the  contractor  must, 
eventually  cone  to  realize  that  he 
will  be  the  loser  if  he  dees  not 
negotiate  prior  to  the  corpletion  of 
the  effort.  At  least  that's  the 
theory. 

For  example,  a  contract  may  have  a 
ceiling  price  rate  of  120%.  This 
percentage  is  applied  against  the 
target  cost  to  develop  the  ceiling 
price.  If  the  charge  effort  is 
similar  in  nature  to  the  basic 
contract,  it  is  reasonable  to  apply 
the  120%  against  the  target  cost  of 
the  change  order.  If  actuals  are 
used  as  the  target  cost,  there  is  no 
justification  for  increasing  the 
contract  ceiling  price  due  to  the 
relative  lack  of  risk  associated  with 
the  change  order  effort.  The 
increase  to  the  contract  ceiling 
price  would  most  likely  be  the  target 
price  (target  cost  plus  target 
profit) . 

This  particular  contractor  did 
eventually  come  to  realize  the  error 
of  his  ways  however  he  was  not 
successful  in  recouping  near  as  many 
ceiling  dollars  as  he  would  have  had 
he  negotiate!  in  a  timely  manner. 

The  second  example  of  an  appropriate 
utilization  of  actual  costs  for  the 
definitization  of  charge  orders 
occurred  during  the  current  time 
period  can  a  contract  for  a  major 
modification  to  a  fielded  piece  of 
equipment.  The  value  of  this 
contract  was  in  excess  of  $10®  and 
this  was  a  fixed  price  development 
contract.  The  use  of  a  fixed  price 
was  deemed  reasonable  for  this  effort 
due  to  the  off-the-shelf  nature  of 
the  acquisition.  (The  contract  was 
initially  awarded  prior  to  the 
revised  policy  on  fixed  price 
development  contracts) . 

As  the  development  progressed,  two 
things  became  apparent;  first,  there 
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was  a  significant  amount  of  software 
development  associated  with  the 
effort.  Second,  and  more  important, 
the  requirements  were  reduced 
substantially,  at  least  in  terms  of 
future  potential. 

These  facts  are  relevant  in  light  of 
the  fact  that  the  contractor 
initially  was  awarded  the  development 
contract  with  what  the  government 
believed  was  a  low  estimate  for 
ocnpletion  of  the  development 
program.  The  production  contract  was 
issued  cxncurrently  with  the  ongoing 
development  contract  and  was  a  sole 
source  fixed  price  incentive 
contract.  There  are  indications  that 
the  contractor  bid  on  the  development 
contract  in  part  to  get  his  "foot  in 
the  door"  for  future  Air  Force 
business. 

It  is  the  contention  of  this  paper 
that  this  contractor  should  be 
treated  situationally,  i.e.  the 
development  contract  is  over-ceiling 
and  negotiations  for  changes  on  this 
contract  should  be  significantly 
different  from  those  on  the 
production  contract.  The  proposals 
from  the  contractor  on  the 
development  contract  consistently 
appeared  to  be  inflated,  while  no 
such  problem  existed  on  the 
production  contract.  Estimates  on 
the  production  contract  were  within 
the  norm  for  negotiation  purposes. 
This  could  perhaps  be  explained  by 
the  fact  that  the  production  contract 
was  in  a  significant  underrun 
situation  while  the  development 
contract  was  over  ceiling. 

The  analysis  of  the  rationale  for  the 
hours  proposed  invariably  determined 
an  unreasonableness  that  was  further 
compounded  by  an  unwillingness  to 
negotiate,  i.e.  little  flexibility  in 
their  position.  Why  was  this?  It 
seems  likely  that  this  was  an  attenpt 
to  regain  ceiling. 

The  data  shews  that  initial  proposals 


were  evaluated  with  major  exceptions 
that  factfinding  was  not  able  to 
resolve.  As  updates  were  presented, 
the  data  would  track  at  or  below  the 
government's  estimates,  however  the 
contractor  would  keep  the  estimate  to 
ccrplete  at  a  level  that  was 
consistent  with  the  original  proposal 
rather  than  that  of  the  actuals  that 
were  being  incurred. 

If  we  evaluate  the  chart  tracking 
this  data  (fig  1)  we  see  a  consistent 
reduction  in  each  of  the  contractor's 
updates,  until  completion  of  the 
effort  which  invariably  would  be  less 
than  the  government's  estimates.  The 
final  settlement  was  consistently 
below  the  government '  s  original 
estimate. 


FIG  1 

C/O  NTE  PROP  OBJ  S/A  NBG 

900,000  600,350  435,250  330,000 
715,000  620,000  515,500  485,000 
675,000  600,000  450,000  353,000 
300,000  250,000  200,000  172,000 


Why  would  this  happen?  I  believe 
there  are  two  reasons  for  the 
predictable  nature  of  the  data. 
First,  the  government  utilizes  the 
contractor's  data  as  a  basis  for 
their  technical  position.  The  lack 
of  history  on  these  types  of  efforts 
complicates  the  analysis  both  by 
government  and  contractor  personnel. 
As  work  is  accomplished,  it  becomes 
less  complicated  and  the  primary 
evaluation  becomes  more  a  question  of 
reasonableness.  The  technical 
reviewer  can  review  incurred  costs, 
utilize  a  modified  cost  performance 
technique  and  develop  an  overall  cost 
to  perform. 

The  proposal  update  data  on  the 
development  contract  did  not  reflect 
this  adjustment.  Some  of  this 
reluctance  could  be  dictated  by  the 
normal  conservatism  of  program 
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management  in  predicting  the  unknown. 
Many  program  managers  cure  hesitant  to 
dip  into  management  reserve  until 
most  if  not  all  tasks  are  completed. 


Over  time,  the  consistent  nature  of 
the  proposals  seems  to  indicate  an 
alternate  agenda.  It  is  not 
unreasonable  to  opine  that  the 
contractor  was  atterrpting  to 
artificially  inflate  the  ceiling 
price  on  the  modification  in  order  to 
recover  dollars  on  the  basic  contract 
that  would  otherwise  need  be  recouped 
from  corporate. 

The  pattern  is  unmistakable.  The 
change  order  would  be  issued  with  an 
inflated  not-to— exceed  (NTE) .  The 
initial  proposal  would  shew  a  large 
decrease  from  the  NTE.  The  technical 
review  would  indicate  a  problem  with 
the  estimating  methodology,  i.e. 
consistently  high  end.  And  the 
government's  objective  would  be 
outside  the  band  of  negotiability. 
Additional  factfinding  would  add  time 
to  the  process  during  which  the 
contractor  would  be  required  to 
provide  updates  containing  actuals 
(due  to  the  Truth  in  Negotiation 
Act) .  These  proposal  updates  world 
trend  downward,  as  would  the 
government's  objective.  Until, 
finally,  the  actual  data  would  result 
in  a  settlement  which  was  not  only 
lower  than  the  contractor's 
proposals,  but  also  significantly 
below  the  initial  estimates  of  the 
government's  technical  team. 

The  eventual  settlements  of  these 
change  orders  contained  markedly 
reduced  increases  in  the  contracts 
ceiling  price.  Had  the  contractor 
merely  accepted  the  government's 
initial  offer,  he  would  have  realized 
a  significantly  greater  increase  in 
the  contract  ceiling  price. 

This  approach  is  obviously  not  to  be 
recommended  for  any  but  the  most 
unique  cases.  It  does  not  motivate 


the  contractor  to  reduce  cost 
incurrence,  but  rather  would  seem  to 
incentivize  him  to  spend 
unnecessarily.  The  data  seems  to 
indicate  that,  even  with  this 
potential,  the  government  is  better 
off  using  actuals  in  these  cases  than 
it  would  be  to  issue  a  unilateral  or 
move  toward  the  contractor '  s 
position. 

I  believe  that  seme  contractors 
realize  the  pressure  that  is  levied 
on  the  government  negotiation  teams 
and  try  to  game  the  process.  They 
believe  that  once  a  change  order  gets 
old  enough,  management  will  pressure 
the  government  to  compromise  their 
position  in  the  hope  of  reaching  a 
settlement.  Once  contractors  cone  to 
realize  that  the  government  team  will 
not  be  pressured,  they  eventually 
revise  their  tactics  and  find 
additional  negotiation  flexibility  in 
their  position. 

A  final  example  of  a  situation  in 
which  the  contractor  attempted  to  use 
the  "changes''  provision  to  recover  a 
negative  financial  position  is  a 
contract  for  software  development  in 
which  the  contractor  accepted  a  lew 
estimated  cost  in  order  to  get  a 
"beachhead"  into  a  weapon  system 
which  had  potential  for  additional 
business  in  the  future.  The  contract 
required  both  software  development 
and  software  maintenance.  As 
problems  surfaced,  the  contractor 
tried  to  have  the  government  revise 
their  estimated  cost  to  reflect  the 
resolution  of  those  problems. 

Under  the  cost  plus  contract,  the 
contractor's  costs  were  all 
recoverable,  however,  the  contractor 
saw  the  changes  as  an  opportunity  to 
adjust  the  award  fee  and  base  fee. 
If  the  exists  could  be  considered  as 
changes,  then  the  fee  would  be 
increased.  Further,  if  the  estimated 
cost  of  the  change  could  be  inflated, 
then  the  award  fee  pool  would  grew  to 
reflect  what  their  management 
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perceived  as  a  more  "reasonable" 
amount. 

While  sate  of  the  "changes"  were 
clearly  within  the  scope  of  the 
contract,  sane  were  additive  in 
nature  and  change  orders  were 
approved  for  these  efforts.  The 
contractor  proposals  for  these 
changes  were  inflated,  based  upon  our 
technical  review.  When  the  actuals 
became  available,  it  was  clear  that 
the  increase  to  the  estimated  cost 
would  be  belcw  even  the  government's 
objective.  The  fee  increase  would 
also  be  less  than  the  basic  contract 
since  the  risk  was  also  reduced. 

OCNGUBICN 

The  basic  premise  that  all  change 
orders  should  be  negotiated  within  a 
180  day  period  needs  to  be  revised. 
There  are  sane  instances  in  which  the 
government  is  best  served  by  allowing 
actuals  to  be  negotiated.  Management 
should  be  more  flexible  in  ad  lowing 
discretionary  authority  to  the 
contracting  office  to  slip  schedule 
on  the  definitization  of  UCA's. 

There  are  instances  in  which  data 
indicates  that  waiting  for  actual 
data  may  be  in  the  best  interests  of 
the  government.  As  contractors  came 
to  accept  that  gamesmanship  of  UCA's 
is  not  in  their  interest  it  is 
possible  that  they  may  be  motivated 
to  became  more  responsive  to 
negotiating  UCA's  even  in  those 
instances  described  above. 
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LESSONS  LEARNED  IN  PAVING  THE  ROAD  TO  QUALITY 
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ABSTRACT 

Vice  President  Gore's  National  Performance 
Review,  published  in  September  1993, 
outlined  twenty  recommendations  for 
reinventing  Government  procurement.  One 
of  those  recommendations  placed  renewed 
emphasis  on  buying  best  value  supplies  and 
services.  Executive  Order  12931  implements 
this  recommendation  by  requiring  the 
government  to  "promote  best  value  rather 
than  simply  low  cost."  This  research  paper 
explores  the  basis  for  confusion  about  the 
best  value  contracting  process  in  light  of  its 
statutory  basis,  acquisition  reform  initiatives, 
and  the  implementation  standards  that 
continue  to  be  defined  by  the  boards  of 
contract  appeal  and  the  courts. 

INTRODUCTION 

Best  value  is  a  quality  focused  approach  to 
government  contracting  which  is  often 
misunderstood  and  inconsistently  applied. 
Traditionally,  the  government  has  based  the 
majority  of  its  contract  awards  solely  on 
price  and  price  related  factors.  In  many 
cases,  however,  the  lowest  priced  product  or 
service  may  prove  to  be  false  economy  if, 
over  the  long  term,  there  are  excessive 
maintenance  costs,  late  deliveries,  or  quality 
defects  which  result  in  greater  overall  costs. 
The  large  federal  deficit  and  shrinking 
defense  dollars  of  recent  years  have  forced 
the  government  to  direct  its  focus  on  ways  to 
get  the  most  value  for  its  money.  Thus,  we 


have  seen  a  growing  shift  to  a  quality  based 
procurement  philosophy.  This  philosophy 
encourages  consideration  of  quality  factors 
in  addition  to  price  alone  and  recognizes  that 
the  government  can  make  tradeoffs  between 
cost  and  quality  factors  and  pay  a  premium 
for  better  quality,  if  it  makes  good  business 
sense  to  do  so.  Even  greater  emphasis  on 
best  value  can  be  expected  in  the  future  as  a 
result  of  the  Acquisition  Streamlining  Act  of 
19941  and  Executive  Order  12931 2  which 
encourage  buying  best  value  supplies  and 
services  and  endorse  the  evaluation  of  past 
contract  performance  as  a  key  quality 
consideration  for  awarding  government 
contracts.  Industry's  concerns  about  the 
largely  subjective  nature  of  the  best  value 
process3  and  the  lack  of  a  commonly 
understood  definition4  have,  however,  led  to 
an  increasing  number  of  protests  in  this  area.5 
The  objective  of  this  paper  is  to  analyze  the 
government's  best  value  process  in  order  to 
provide  a  better  understanding  of  some  of  its 
most  difficult  issues  through  a  discussion  of 
(a)  pertinent  statutes  and  regulations;  (b)  the 
impact  of  acquisition  reform;  (c) 
implementation  concerns;  and  (d)  lessons 
learned  from  selected  protests. 

STATUTE  AND  REGULATION 

The  Competition  in  Contracting  Act 
(CICA)  of  1984  established  the  foundation 
for  the  best  value  approach  through  its 
prescribed  procedures  for  the  planning, 
solicitation,  evaluation,  and  award  of 
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competitive  government  contracts.6  In 
amending  the  civilian  and  defense  agency 
procurement  statutes,7  the  CICA's  primary 
purpose  was  to  place  severe  restrictions  on 
the  government's  use  of  noncompetitive 
procedures  and  thereby  enhance 
opportunities  for  all  responsible  sources  to 
compete  for  government  contracts.  The 
CICA  prescribes  two  basic  methods  for 
obtaining  competition,  sealed  bidding  and 
competitive  proposals.  The  Act  directs  the 
government  to  select  the  approach  that  best 
fits  the  acquisition  needs.8  However,  it  also 
reflects  a  statutory  preference  for  sealed 
bidding  when  all  of  the  following  criteria  are 
present: 

(i)  time  permits  the  solicitation, 
submission,  and  evaluation  of  sealed 
bids; 

(ii)  the  award  will  be  made  on  the  basis 
of  price  and  other  price-related  factors; 

(iii)  it  is  not  necessary  to  conduct 
discussions  with  the  responding  sources 
about  their  bids;  and 

(iv)  there  is  a  reasonable  expectation  of 
receiving  more  than  one  sealed  bid.9 

Under  the  sealed  bidding  procedure,  the 
government  broadly  publicizes  the 
solicitation  for  competitive  sealed  bids, 
announces  prices  received  at  public  bid 
openings,  compares  only  prices  and 
price-related  factors  of  bidders  who  meet  the 
clear  requirements  laid  out  in  the 
specifications,  and  awards  without  discussion 
to  the  responsible  bidder  whose  bid  meets  all 
the  requirements  of  the  solicitation  at  the 
lowest  price.10  The  statutory  preference  for 
this  method  is  based  on  the  implicit  fairness 
and  simplicity  of  the  sealed  bidding  process 


and  the  assumption  that  when  the  criteria  for 
sealed  bidding  exist,  the  best  value  can  be 
determined  by  a  straightforward  comparison 
of  prices. 

The  statute  permits  the  government  to  use 
the  competitive  proposals  procedure 
(commonly  known  as  competitive 
negotiations)  only  when  it  can  show  that 
sealed  bidding  is  not  appropriate  in  light  of 
the  four  specified  criteria.  Under  the 
competitive  proposals  procedure,  the 
government  may  conduct  discussions  with 
the  offerors  and  factor  non  cost  related 
quality  factors  in  with  cost  in  making  the 
award  decision.  The  competitive  proposals 
procedure  would  normally  be  used  when  the 
government  has  a  need  to  seek  out  quality 
which  cannot  be  determined  by  price  alone 
and  is  willing  to  pay  more  for  that  quality. 
The  General  Accounting  Office  (GAO)  has 
generally  supported  government  discretion 
to  use  competitive  negotiations  unless  its 
intent  is  to  award  solely  on  the  basis  of  price 
and  it  cannot  demonstrate  a  need  for 
technical  proposals."  Thus,  in  the  recent 
case  of  Premier  Vending,  the  GAO  upheld 
the  government's  decision  to  use  negotiations 
where  a  comparative  evaluation  of  technical 
evaluation  factors  was  warranted.12  In 
RaycaJ  Filter  Technologies,  however,  the 
Army  had  used  competitive  negotiations  but 
based  the  award  solely  on  price  and  had  no 
need  for  and  did  not  request  technical 
proposals.  The  competitive  negotiations 
procedure  was  essentially  used  as  an 
administrative  convenience  to  facilitate  the 
government's  ability  to  deal  with  changes  in 
requirements.  GAO  held  that  the  possibility 
of  changes  was  not  in  and  of  itself  adequate 
basis  to  permit  negotiations  and  sustained  the 
protest  upon  finding  that  the  four  statutory 
factors  for  sealed  bidding  were  present.13 
These  cases  illustrate  that  despite  the  current 
emphasis  on  best  value,  government  agencies 
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still  need  to  carefully  analyze  each  individual 
requirement  in  light  of  the  statutory  criteria 
for  sealed  bidding  before  using  a  competitive 
negotiation  approach. 

To  promote  fairness  in  competition,  the 
CICA  also  mandates  that  government 
solicitations  use  nonrestrictive  specifications 
and  clearly  inform  offerors  of  the 
groundrules  (cost  factors,  non  cost  factors, 
and  their  relative  importance)  that  the 
government  will  use  in  making  the  selection 
decision. 

The  Federal  Acquisition  Regulation 
(FAR)  implements  the  statutory 
requirements  of  CICA,  but  gives  little  further 
procedural  guidance.  The  FAR  coverage  on 
source  selection  in  Subpart  15.602  describes 
the  two  approaches  for  conducting 
competitive  negotiations  as  follows; 

(1)  Cost  or  price  competition  between 
proposals  that  meet  the  government's 
minimum  requirements  stated  in  the 
solicitation;  or 

(2)  Competition  involving  an  evaluation 
and  comparison  of  cost  or  price  and 
other  factors.14 

The  first  approach  is  commonly  known  as 
the  Low  Price  Technically  Acceptable 
(LPTA)  approach.  This  involves  a  pass/fail 
evaluation  of  each  offeror's  technical 
proposal  against  the  solicitation's  minimum 
requirements,  a  comparison  of  the  prices 
submitted  by  all  offerors,  and  award  to  the 
technically  acceptable  responsible  offeror 
submitting  the  lowest  evaluated  price.  This 
is  the  simplest  approach  to  competitive 
negotiations.  It  is  generally  used  when  the 
government  needs  more  information  to 
determine  technical  acceptability  but  also  has 
confidence  that  the  low  offer  will  yield  the 
best  value  for  relatively  straightforward 


requirements  with  no  prior  history  of  poor 
performance.  The  second  approach  is  what 
is  commonly  referred  to  as  the  best  value 
approach.  Under  this  approach,  the 
government  not  only  evaluates  the  technical 
proposals  but  also  compares  the  strengths, 
weaknesses  and  risks  of  the  competing 
proposals  in  terms  of  both  the  cost  and  non 
cost  or  quality  factors  stated  in  the 
solicitation.  The  best  value  approach  gives 
the  government  greater  flexibility  and 
discretion  to  make  tradeoffs  between 
cost/price  and  quality  factors  such  as 
technical  approach,  management  capabilities, 
and  past  performance  in  selecting  the 
proposal  offering  the  best  combination  of 
quality  factors  and  cost.  Under  this 

approach,  the  government  is  not  bound  to 
award  to  the  lowest  price  offeror.  Although 
the  FAR  does  not  explicitly  define  best 
value,  it  comes  closest  to  describing  the 
concept  at  15.605(c)  where  it  states: 

While  the  lowest  price  or  lowest  total 
cost  to  the  Government  is  properly  the 
deciding  factor  in  many  source 
selections,  in  certain  acquisitions,  the 
Government  may  the  select  the  source 
whose  proposal  offers  the  greatest 
value  to  Government  in  terms  of 
performance  and  other  factors. 

PUBLIC  VS  PRIVATE  SECTOR 
APPLICATION 

Best  value  is  a  concept  that  is  regularly  used 
by  the  private  sector  when  making  major 
purchases.  In  the  case  of  purchasing  an 
automobile,  a  private  consumer  will  rarely 
base  the  buying  decision  solely  on  price,  but 
rather  will  weigh  the  cost  against  other  more 
subjective  quality  factors  such  as  handling, 
comfort,  design,  maintainability,  and  the 
customer  service  record  of  the  dealer.  In  the 
private  sector,  the  process  is  relatively 
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simple,  unencumbered  by  rigid  rules  and  not 
subject  to  protest  from  unsuccessful 
competitors.  Commercial  companies,  in 
making  buying  decisions,  are  free  to  limit 
competition  to  a  handful  of  good  suppliers 
with  whom  they  often  establish  long  term 
relationships.  As  normal  practice,  they 
openly  communicate  with  potential  suppliers 
during  all  phases  of  the  process  and  place  a 
major  emphasis  on  the  record  of  past 
performance.1'’  In  contrast,  the  government 
process  has  become  both  complex  and 
costly.  This  is  because  of  the  various 
statutes  and  regulations  which  mandate  equal 
treatment  and  competition  among  all  possible 
vendors.  These  rules,  unique  to  government 
contracting,  require  consideration  of  social 
and  economic  issues  and  place  severe 
limitations  on  any  dialogue  between  the  user 
and  supplier  during  the  solicitation  phase.16 
This,  and  the  risk  of  protest  from 
unsuccessful  competitors,  contributes  to  the 
difficulties  faced  by  the  government 
purchaser  in  reasonably  applying  subjective 
judgment  while  protecting  the  rights  of  all 
sellers  to  compete  on  an  equal  basis  for 
expenditure  of  public  funds. 

THE  IMPACT  OF  ACQUISITION 
REFORM 

While  the  best  value  concept  is  not  new  to 
the  government,17  recent  efforts  to  streamline 
the  government  procurement  system  have 
resulted  in  a  new  emphasis  on  best  value 
buying.  This  new  high  level  emphasis, 
embodied  in  Executive  Order  1293 118, 
implements  the  following  recommendations 
of  the  National  Performance  Review  (NPR) 
and  acquisition  reform  legislation  of  the 
103rd  Congress. 


THE  NATIONAL  PERFORMANCE 
REVIEW 

Among  the  20  recommendations  for 
"Reinventing  Federal  Procurement" 
contained  in  the  1993  National  Performance 
Review  was  a  recommendation  encouraging 
best  value  procurement.19  This 

recommendation  suggested  that  the 
government  recognize  quality  factors  in 
addition  to  price;  define  the  term  "best 
value";  and  develop  "best  practices" 
guidance  to  facilitate  the  use  of  best  value 
source  selection  procedures. 

ACQUISITION  REFORM 
LEGISLATION 

The  reforms  contained  in  the  Acquisition 
Streamlining  Act  of  1994  implement  the 
recommendations  in  the  National 
Performance  Review.  This  Act  amends  the 
solicitation,  evaluation,  and  award 
requirements  of  CICA  by  requiring  that  the 
Armed  Services  and  civilian  agencies  clearly 
disclose  to  competitors  not  only  the 
evaluation  factors  but  also  the  subfactors 
that  will  impact  the  award  decision.  The  Act 
also  specifically  recognizes  past  performance 
as  a  quality  evaluation  factor,  mandates  that 
cost  or  price  be  considered  in  the  evaluation, 
and  requires  that  solicitations  disclose  the 
relative  importance  of  cost  or  price  to  all  non 
cost  factors  combined.20  With  regard  to  past 
performance,  the  Act  formalizes 
congressional  recognition  of  an  offeror's  past 
contract  performance  as  an  important 
indicator  of  an  offeror's  potential  to 
successfully  perform  future  contracts.  The 
Act  further  requires  the  government  to 
develop  clear  guidelines  that  will  assist 
agencies  in  their  evaluation  of  contractor 
past  performance.21 
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PAST  PERFORMANCE  INITIATIVE 

The  FAR  never  prohibited  the  government 
from  evaluating  past  performance.  In  a  1990 
study,  however.  Dr.  Stephen  Kelman,  a 
Harvard  professor  of  public  policy,  found 
that  most  government  buyers  of  complex 
computer  systems  considered  themselves 
forbidden  to  make  comparative  evaluations 
of  competing  offerors'  past  performance 
records  when  making  decisions  to  award 
new  contracts.22  He  further  found  that  this 
belief  evolved  in  the  procurement  culture 
from  "doctrine  that  vendors  may  be 
evaluated  only  on  their  proposals"  and  not 
on  extrinsic  data.23  Under  such  a  system,  he 
reasoned,  there  is  little  incentive  for  suppliers 
to  do  a  good  job  once  they  win  a  contract.24 
His  concluding  recommendations  included 
making  past  performance  a  formal  evaluation 
factor25  and  dramatically  increasing  public 
officials'  discretion  to  use  their  own 
judgment  in  evaluating  past  performance  in 
selecting  contract  sources.26 

Since  Kelman's  confirmation  in  November  of 
1993  as  the  seventh  administrator  of  the 
Office  of  Federal  Procurement  Policy 
(OFPP),  he  has  worked  to  make  the 
consideration  of  past  performance  in  award 
decisions  a  top  priority.27  Almost  a  year 
before  Kelman's  appointment,  in  January 

1993,  the  OFPP  issued  past  performance 

policy  in  Policy  Letter  92-5. 28  The  new 

policy  required  government  agencies  to 
collect  past  performance  information  on  its 
contracts  and  to  use  that  information  in 
evaluating  competitively  negotiated  offers 
over  $100,000.  One  of  Kelman's  first  major 
efforts  as  OFPP  administrator  was  to  launch 
a  pilot  program  to  foster  use  of  past 
performance  in  source  selections  and  to 
facilitate  implementation  of  the  policy.29 
Under  the  pilot  program,  initiated  in  January 

1 994,  the  OFPP  and  twenty  federal  agencies 


signed  voluntary  pledges  to  make  past 
performance  a  significant  factor  in  selecting 
contractors  on  sixty-one  designated 
contracts  scheduled  for  award  over  an 
eighteen  month  period.  The  experiences 
gained  during  the  pilot  will  be  key  to 
development  of  the  past  performance 
guidance  directed  by  the  1994  acquisition 
reform  legislation.30 

IMPLEMENTATION 

CONCERNS 

Increasing  use  of  the  best  value  approach  has 
raised  concerns  in  both  the  private  and  public 
sectors  about  its  effective  implementation. 

INDUSTRY.  In  November  1993,  the 
Information  Technology  Association  of 
America  (ITAA)  issued  a  position  paper  on 
the  use  of  the  best  value  approach  in 
government  acquisitions  of  information 
technology.  A  summary  appearing  in 
Federal  Contracts  Report 31  indicated  that 
ITAA,  while  supporting  use  of  the  best  value 
concept  for  complex  acquisitions, 
emphasized  the  need  for  a  number  of 
changes  in  the  way  in  which  the  government 
conducts  its  best  value  acquisitions.  Some  of 
the  recommended  changes  included 
adequately  defining  government  needs  in 
functional  terms  to  encourage  innovative 
offeror  solutions;  communicating  as  much 
information  as  possible  to  offerors  about 
mission  objectives  and  evaluation  factors 
early  in  the  process;  and  ensuring  that 
government  officials  are  adequately  trained 
in  making  cost/technical  tradeoff  analyses 
and  judgmental  business  decisions.  In  the 
services  contracting  arena,  concerns  have 
also  been  voiced  by  the  Professional  Services 
Council.  In  its  proposal  for  reform,  the 
Council  advocates  use  of  a  best  value 
approach  but  also  notes  that  the  government 
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often  tends  to  place  too  much  emphasis  on 
cost  at  the  expense  of  quality.32  In  the 
accompanying  working  paper  on  best  value 
procurement,  the  Council  states  that 
although  some  agencies  attempt  to  use  a  best 
value  approach,  often  "these  efforts  lack  the 
rigor  and  procedural  discipline  needed  to 
avoid  excessive  price/cost  bias  in  source 
selection."3'' 

DEPARTMENT  OF  DEFENSE  (DOD). 

Industry  concerns  are  echoed  in  a  June  1994 
DOD  Inspector  General  (IG)  report  on  a 
sampling  of  19  formal  source  selections 
conducted  by  various  DOD  activities 
between  1989  and  1991. 34  The  findings 
identified  a  number  of  systemic  problems 
associated  with  DOD's  implementation  of  the 
best  value  concept.  One  of  the  key  problem 
areas  was  confusion  over  the  definition  of 
"best  value".  In  the  course  of  interviewing 
56  program  managers,  evaluators,  and 
contracting  officers,  the  IG  found  no 
certainty  or  consistency  in  "what  constituted 
a  'Best  Value'  determination".35  Other 
concerns  highlighted  problems  with  poorly 
defined  evaluation  criteria,  too  many 
evaluation  criteria  and  vague  and 
inconsistent  solicitation  language. Jb 
Although  most  of  the  programs  in  the  survey 
used  about  20  evaluation  criteria,  one  of  the 
major  programs  surveyed  used  645,  another 
used  90,  and  a  third  used  73. ,7  Excessive 
numbers  of  evaluation  criteria  make 
evaluation  overly  complex  and  dilute  the 
government's  ability  to  effectively 
discriminate  among  the  offerors.  The  report 
also  cited  inadequate  understanding  of  the 
difference  between  the  Low  Price 
Technically  Acceptable  (LPTA)  and  best 
value  approaches.  This  was  reflected  in 
many  of  the  solicitations  in  the  survey  that 
stated  award  would  be  made  on  the  basis  of 
best  value  but  where  other  language  in  the 


same  solicitation  actually  described  a  LPTA 
approach.38  Recommendations  for 

improvement  addressed  establishing 
consistency  in  the  execution  of  best  value 
contracts,  streamlining  the  process,  and 
ensuring  adequate  training  of  those  who 
participate  in  the  process.39 

LESSONS  LEARNED  IN 
SELECTING  FOR  QUALITY 

The  critical  questions  that  government 
activities  must  address  in  making  best  value 
decisions  are  what  constitutes  "quality"  and 
how  do  you  measure  the  value  of  that 
quality.  In  making  the  best  value  award 
decision,  the  government  has  broad 
discretion  as  long  as  the  decision  is  rational, 
consistent  with  the  evaluation  criteria,  and 
well  documented.40  The  evaluation  criteria 
and  their  relative  importance  play  a  major 
role  in  any  cost/technical  tradeoff  analysis. 
Ultimately,  however,  the  selection  decision 
must  revolve  on  the  significance  of  the 
differences  between  proposals  and  to  what 
extent  the  government  is  able  to  demonstrate 
whether  the  superior  quality  of  a  particular 
proposal  warrants  the  difference  in  cost.4' 

MEASURING  THE  VALUE  OF 
QUALITY 

Some  recent  cases  illustrate  a  growing  trend 
for  contract  appeal  boards  to  demand  a  more 
exacting  measurement  of  value  to  justify 
payment  of  a  price  premium.  This  may  mean 
quantifying  the  value  of  the  benefits  as 
reflected  in  the  following  cases. 

Justifying  the  Ultimate  Price  Premium. 

The  Federal  Circuit  Court  of  Appeals 
decision  to  uphold  the  $1.4  billion  dollar 
award  to  AT&T  for  the  Treasury, 
Multi-User  Acquisition  Contract  (TMAC)  is 
perhaps  the  most  dramatic  best  value 
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decision  to  date.  The  TMAC  requirement 
covered  office  automation  systems  and 
associated  software,  maintenance  and 
support  services  for  the  Internal  Revenue 
Service  (IRS).  The  evaluation  factors  were 
broadly  crafted  stating  in  part. 

Award  will  be  made  to  that  offeror 
whose  proposal,  containing  the 
combination  of  technical,  management 
and  support,  and  cost  features,  offers 
the  best  overall  value  to  the 
government.  The  Government  is  more 
concerned  with  obtaining  superior 
technical  (technical  and  management 
and  support)  features  than  making 
award  to  the  lowest  cost.  Award  will 
be  determined  by  comparing  differences 
in  the  value  of  technical  features  with 
difference  in  overall  cost  to  the 
Government...42 

IRS  awarded  the  seven  year  contract  to 
AT&T  at  an  evaluated  price  of  $1.4  billion. 
Lockheed  and  IBM,  whose  evaluated  prices 
were  $900  million  and  $700  million, 
respectively,  protested  the  award  to  the 
General  Services  Board  of  Contract  Appeals 
(GSBCA),  arguing  that  the  award  was  not 
proper  in  light  of  the  much  higher  cost  of  the 
selected  proposal.  The  Board  sustained  the 
protests,  finding  that  there  was  "nothing  in 
the  record  to  explain  how  AT&T's  technical 
advantage  justified  the  price  premium  of 
one-half  to  three-quarters  of  a  billion  dollars 
that  the  IRS  was  willing  to  pay."43  The 
board  returned  the  action  to  the  IRS  for  a 
more  detailed  evaluation  after  concluding 
that  the  IRS'  cost/technical  tradeoff  analysis 
was  not  meaningful.  The  IRS  then 
performed  an  extensive  analysis  in  which 
they  quantified  in  terms  of  dollars  some  of 
the  proposal  differences  considered  most 
likely  to  impact  IRS  operations.  As  a  result, 
the  IRS  again  concluded  that  the  value 
offered  by  the  AT&T  proposal  warranted  the 


price  premium.  Lockheed  and  IBM 
protested  once  more  to  the  GSBCA.44  In 
reviewing  the  IRS's  more  detailed  analysis, 
the  GSBCA  looked  at  the  methodology  used 
to  perform  the  cost/technical  tradeoff.  This 
approach  included  the  use  of  discriminators, 
drawn  from  the  strengths,  weaknesses,  and 
risks  of  the  proposal  evaluation,  which  were 
determined  to  (1)  reflect  significant  areas  of 
differences  between  the  proposals;  (2)  be 
important  to  the  objectives  of  the  program; 
and  (3)  have  an  impact  on  IRS  operations.45 
The  IRS  divided  the  discriminators  into  4 
quantified  areas  and  10  non-quantified 
impact  areas.  The  differences  in  the  four 
quantified  areas  were  determined  to  have  a 
direct  relationship  to  user  productivity  or 
price  risk  and  were  translated  into  dollars. 
These  areas  addressed  the  impact  of 
technology  on  contract  price  risk;  the  impact 
of  performance  on  mission  productivity;  the 
impact  of  software  integration  on  end-user 
productivity;  and  the  impact  of  training  on 
end-user  productivity.  Although  the 
protesters  challenged  the  IRS's  assumptions 
and  their  choice  of  discriminators,  the  board 
supported  both  the  method  used  in 
performing  the  cost/technical  tradeoff  and 
the  IRS'  decision  because  they  were  found  to 
be  reasonable  and  consistent  with  the 
solicitation  criteria.  The  board  emphasized 
that  an  agency  has  a  great  deal  of  discretion 
in  conducting  the  tradeoff  analysis  as  long  as 
it  is  reasonable  and  demonstrates  that  the 
expected  benefits  are  worth  the  premium  in 
price.  Lockheed  argued  that,  based  on  the 
fact  that  only  four  areas  were  selected  for  a 
quantitative  analysis,  it  would  have  had  to 
offer  its  products  at  no  charge  in  order  to  get 
the  award.  In  response,  the  Board  stated, 


"...we  perceive  no  reason  why  goods 
and  services  cannot  have  a  value  to 
their  purchaser  that  exceeds  their 
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price.  One  has  only  to  think  of  an 
automobile  air  bag."45 

The  board  also  noted  that  none  of  the 
competitors  had  protested  the  broadly  stated 
evaluation  provisions  prior  to  receipt  of 
initial  proposals.  Absent  any  objection  to  the 
groundrules  laid  out  in  the  solicitation,  the 
IRS  had  considerable  discretion  in  selecting 
the  method  used  to  compare  proposals  and 
was  found  not  to  have  abused  that  discretion 
in  conducting  its  subsequent  cost/technical 
tradeoff  analysis.47 

Lockheed's  appeal  to  the  U.S.  Court  of 
Appeals  for  the  Federal  Circuit  was  also 
denied.48  In  its  decision,  the  Court  cited 
numerous  cases  which  support  the 
government's  broad  discretion  in  making  the 
best  value  decision.  It  further  found  that  the 
productivity  dollar  value  of  the  AT&T 
solution  far  outweighed  that  of  the  other 
proposals  although,  in  terms  of  point  scores, 
this  represented  less  than  1 5%  of  all  available 
points.  Thus,  it  noted  that 

...a  proposal  which  is  one  point  better 
than  another  but  costs  millions  of 
dollars  more  may  be  selected  if  the 
agency  can  demonstrate  within  a 
reasonable  certainty  that  the  added 
value  of  the  proposal  is  worth  the 
higher  price.49  (emphasis  in  original) 

In  this  case,  the  Court  supported  the  IRS  in 
its  selection  because  it  was  able  to  clearly 
demonstrate,  by  quantifying  some  of  the  key 
discriminators,  that  the  added  benefits  were 
worth  the  added  cost. 

Payment  of  Price  Premium  More  Closely 
Scrutinized  Where  Technical  Differences 
Are  Slight 

In  a  case  where  the  evaluation  reflected  no 
significant  weaknesses  in  the  closely  rated 


technical  proposals,  the  Board  sustained  the 
protest  because  the  record  did  not 
adequately  show  that  the  value  of  the 
benefits  to  be  derived  from  the  slightly  higher 
rated  offerors  justified  the  higher  prices.50 
The  protest  was  filed  by  B3H  Corporation  in 
response  to  best  value  contract  awards  made 
by  the  Air  Force  to  two  small  businesses. 
The  solicitation  involved  a  requirement  to 
provide  technical  support  services  for 
communication/computer  systems  (C-CS)  as 
part  of  an  effort  to  modernize  the  Air  Force's 
logistics  program.  The  evaluation  criteria 
outlined  in  the  solicitation  were  technical, 
management,  and  cost.  Technical  and 
management  were  specified  as  being  of 
greater  importance  than  cost.  The 
solicitation  also  stated  that  the  government 
would  evaluate  proposal  risk,  performance 
risk,  and  cost  realism.  The  Air  Force  used  a 
color  rating  system  which  graded  proposals 
as  blue,  green,  yellow,  or  red,  with  blue 
being  best.  The  two  awardees  and  the 
protester  all  scored  in  the  blue  and  green 
zones,  had  low  performance  and  proposal 
risks,  and  no  significant  weaknesses.51 

In  evaluating  costs,  the  government 
compared  the  offerors'  proposed  costs  for 
performing  the  sample  statement  of  work  to 
the  government's  estimate  for  performing  the 
same  task.  The  GSBCA  found  this 
methodology  to  be  defective  because  it  did 
not  take  into  account  the  offerors'  varying 
approaches  and  thus  leveled  the  differences 
between  proposals.52 

The  Air  Force  also  performed  an  impact 
analysis  to  determine  which  comparative 
differences  between  proposals  would 
significantly  impact  the  overall  value  to  the 
government.  Using  a  methodology  similar  to 
that  used  in  the  TMAC  case,  the  agency 
identified  seven  discriminators  for  further 
analysis.  However,  only  one  of  the 
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discriminators  was  subject  to  quantification. 
The  selecting  official  used  the  results  of  the 
impact  analysis  to  look  behind  the  relatively 
small  technical  and  management  differences. 
The  source  selection  authority  (SSA) 
concluded  that  the  awardee's  management 
strengths,  as  reflected  in  the  greater 
efficiency  and  overall  quality  of  software 
development,  provided  a  value  that 
warranted  payment  of  a  price  premium  of 
approximately  15%.53  In  spite  of  the  detail 
of  the  impact  analysis,  the  board  criticized 
the  fact  that  the  Air  Force  had  only 
quantified  one  of  the  seven  discriminators 
finding  that, 

The  record  does  not  associate  any 
specific  monetary  value  with  the 
advantages  of  the  three  proposals  to  the 
agency  in  any  greater  detail  and  in  a 
more  convincing  fashion  than  the  broad 
generalities  and  simple  conclusions  of 
the  SSA.54 

The  board  also  played  down  the  SSA's 
concern  about  possible  cost  overruns 
associated  with  the  protester's  proposal, 
indicating  that  a  task  order  contract  allows 
the  government  a  mechanism  for  controlling 
and  distributing  risks  to  perform  certain 
tasks. 

The  board,  in  finding  a  lack  of  clear  evidence 
that  the  differences  in  the  substantially 
similar  proposals  were  worth  the  added  cost, 
stated, 

Although  mathematical  precision  or 
absolute  dollar  quantifications  are 
neither  expected  nor  required,  more 
than  a  mere  conclusion  is  required  to 
justify  a  best  value  analysis."55 

The  board  further  found  that  in  this  case  a 
particularly  strong  justification  was  needed 
since  the  use  of  a  task  order  contract  gave 


the  agency  more  discretion  to  control 
contractor  performance  through  the 
placement  of  individual  task  orders.  Under 
such  circumstances,  and  especially  where 
technical  differences  are  small  or  where  cost 
differences  are  large,  the  greater  the  need  for 
a  more  in-depth  tradeoff  analysis  to  justify 
the  best  value  decision. 

It  is  interesting  to  note  that  a  strong 
dissenting  opinion  was  filed  by  Judge  Donald 
W.  Devine  based  on  his  assessment  that  the 
Board  had  interjected  its  judgment  for  the 
SSA  in  an  area  which  should  be  within  the 
discretion  of  the  decision  making  official.  In 
his  opinion,  he  noted  that  the  board  had 
ignored  extensive  testimony  of  the  SSA 
which  showed  that  the  price  premium  was 
justified  in  light  of  the  agency's  prior 
experience  with  these  requirements  and  the 
awardees'  better  qualifications  which  would 
improve  productivity  and  task  performance. 
Further,  he  stated  that  contract  appeals 
boards  are  not  qualified  to  judge  whether  a 
particular  company's  qualifications  may  or 
may  not  be  worth  the  additional  cost  in 
providing  specific  contract  services  with 
which  only  the  agency  is  intimately  familiar. 
Noting  that  the  difference  between  the  two 
awardees'  costs  and  the  protester's  costs 
were  only  15%  and  5%  respectively,  he 
wrote, 

Under  such  circumstances,  the  Board 
should  never  intrude.  In  this  decision, 
the  Board  has  become  the  Omniscient 
Source  Selection  Authority  that  it  was 
never  meant  to  be.56 

In  spite  of  Judge  Devine's  dissenting  opinion, 
the  Air  Force's  decision  was  overturned  by 
the  board  because  it  lacked  sufficiently  clear 
measurement  of  the  value  of  the  small 
technical  differences  necessary  to  justify  the 
added  cost. 
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EVALUATION  CRITERIA  AS 
GUIDELINES  FOR  THE  SELECTION 
DECISION 

The  selection  decision  must  be  made  within 
the  framework  of  the  stated  evaluation 
criteria.  However,  the  selection  official  has 
considerable  discretion  in  making  judgments 
about  the  significance  of  differences  in 
competing  proposals  and  in  weighing  quality 
factors  against  cost.  The  following  cases 
illustrate  that  the  relative  importance  of  the 
evaluation  criteria  in  the  solicitation  are 
merely  broad  guidelines,  not  rigid  and 
limiting  rules  for  the  selection  decision. 
Thus,  even  where  the  evaluation  criteria  in 
the  solicitation  state  that  the  technical  factor 
is  more  important  than  cost,  the  boards  have 
supported  award  to  a  lower  priced  offeror  if 
the  slight  technical  differences  do  not 
support  payment  of  the  price  premium.  In 
TRW,  Inc.,  the  GSBCA  supported  the  Air 
Force's  decision  to  award  to  a  lower  priced 
offeror  where  the  slight  technical  superiority 
of  the  protester's  offer  did  not  warrant  an 
$11.7  million  price  premium."’7  In  this  case, 
TRW  had  argued  that  the  Air  Force's 
decision  to  award  a  large  information 
technology  technical  support  contract  to 
CSC  was  flawed  because  the  Air  Force  had 
not  quantified  the  technical  advantages  of 
TRW's  proposal  in  making  the  cost/technical 
tradeoff.  The  GSBCA  rejected  TRW's 
argument  finding  that,  where  technical 
differences  between  competing  proposals  are 
slight,  the  government  is  not  required  to 
quantify  the  differences  if  they  do  not 
warrant  payment  of  a  large  price  premium. 
Similarly,  in  Sherikon,  Inc.,  the  GAO 
supported  the  Navy's  decision  to  award  to  a 
lower-priced  offeror  where  the  6  point 
difference  in  technical  ratings  was  found  to 


be  not  significant  enough  to  warrant  a  12% 
premium  in  price. 78 

In  Brunswick  Defense ,  where  price  was  the 
most  important  evaluation  factor,  the  board 
upheld  the  discretion  of  the  selection 
authority  to  award  to  a  higher  priced  offeror 
where  the  record  showed  clear  evidence  that 
the  low  priced  offeror  presented  an 
unacceptable  performance  risk.79  In  this 
case,  which  involved  a  Naval  Air  Systems 
Command  requirement  for  target  missiles, 
the  solicitation  stated  that  price  was 
significantly  more  important  than  the 
technical  factor  which  was  moderately  more 
important  than  the  past 

performance/systemic  improvement  factor 
The  solicitation  also  put  offerors  on  notice 
that  the  government  was  not  bound  to  award 
to  the  lowest  price  offeror  if  the  technical 
advantages  of  a  higher  priced  offeror 
justified  payment  of  the  price  premium.60 
Brunswick's  proposed  price  of  $73  million 
was  approximately  18%  lower  than  Beech's 
price  of  $91  million.  With  regard  to  the 
technical  factor,  both  proposals  were  rated 
as  satisfactory  overall  with  Beech's  proposal 
reflecting  low  risk  and  Brunswick's  medium 
risk.  In  the  area  of  past 

performance/systemic  improvement.  Beech 
received  a  satisfactory/low  performance  risk 
rating  while  Brunswick  was  rated  as 
unacceptable  with  high  performance  risk  due 
to  a  series  of  previously  unsuccessful  missile 
launches.61  After  extensive  discussions  in 
which  Brunswick  was  given  the  opportunity 
to  explain  past  performance  failures  and 
plans  for  corrective  action,  the  Navy 
concluded  that  Brunswick  still  presented  a 
high  performance  risk.  The  Navy  then 
quantified  missile  replacement  costs  and 
range  operation  costs  resulting  from 
Brunswick's  past  record  of  50%  reliability 
rate  compared  with  Beech's  96%  reliability 
rate.  As  a  result,  the  Navy  found  that  award 
to  Brunswick  would  result  in  significant 
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additional  costs  ($33  million).62  Although 
the  selection  official  made  the  decision  to 
award  to  Beech  on  September  28,  the  award 
was  delayed  until  November  5  as  a  result  of 
internal  management  briefings.  On 
November  1,  Brunswick  made  its  first 
successful  missile  launches.  The  selection 
official  learned  of  the  launches  on  November 
3,  but  did  not  reopen  discussion  as  the  award 
decision  had  already  been  made.  The 
protester  argued  that  the  Navy  should  have 
reopened  the  evaluation  after  learning  of 
Brunswick's  four  successful  flights  just  days 
before  the  award.  GAO  disagreed,  finding 
that  the  government  did  not  have  to  delay 
award  to  enable  the  protester  to  bring  its 
performance  record  up  to  an  acceptable  level 
since  the  selection  official  had  already  made 
the  selection  decision  a  month  earlier.63 
Citing  Grey  Advertising,  Inc.,  55  Comp  Gen 
1111  (1976),  76-1  CPD  H325,  GAO  found 
that 

source  selection  officials  in  negotiated 
procurements  have  broad  discretion  in 
determining  the  manner  and  extent  to 
which  they  will  make  use  of  the 
technical  and  cost  evaluation  results. 
Agencies  may  make  cost/technical 
tradeoffs  in  deciding  between 
competing  proposals;  the  propriety  of 
such  a  tradeoff  turns  not  on  the 
difference  in  technical  scores  or  ratings, 
per  se,  but  on  whether  the  selection 
official's  judgment  concerning  the 
significance  of  that  difference  was 
reasonable  and  adequately  justified  in 
light  of  the  evaluation  scheme.64 

SUMMARY 

Protest  statistics  confirm  many  of  the 
problems  associated  with  implementing  the 
best  value  approach.  Protests  related  to 
improper  evaluation  and  negotiation  are 
sustained  for  several  major  recurring 


reasons.  These  include  failure  to  follow  the 
stated  evaluation  criteria;  failure  to  conduct 
meaningful  discussions;  failure  to  analyze 
cost  realism  properly;  and  failure  to  conduct 
proper  cost/technical  tradeoff  analyses.65 
Such  problems  emphasize  the  importance  of 
a  well  planned  acquisition  process  which 
begins  with  converting  customer  needs  and 
performance  objectives  to  performance 
based  requirements;  clearly  communicating 
those  needs  and  criteria  for  quality  to 
industry  in  the  solicitation;  properly 
evaluating  proposal  risk,  cost  realism,  and 
performance  risk  consistent  with  the 
evaluation  criteria;  conducting  meaningful 
discussions;  and  making  a  rational,  well 
documented  award  decision.  The  boards 
and  courts,  as  protest  forums,  will 
undoubtedly  continue  to  define  the  level  of 
discretion  of  the  selection  official  and  the 
degree  of  justification  for  cost/technical 
tradeoffs  in  the  best  value  process. 

CONCLUSION 

The  best  value  product  or  service  in  federal 
acquisition  can  be  achieved  by  traveling  one 
of  several  roads.  Both  the  sealed  bidding 
and  competitive  negotiation  routes  will  lead 
to  quality  under  the  appropriate 
circumstances.  Misunderstandings  arise 
because  best  value  has  become  a  term  of  art 
for  the  competitive  negotiation  procedure 
where  we  consider  factors  in  addition  to 
price.  Compounding  the  confusion  is  the 
fact  that  there  have  been  few  road  maps 
describing  the  most  appropriate  routes  for 
the  journey  and  potential  detours  along  the 
way.  Clearly,  the  sealed  bid  route  is  the 
most  straightforward  and  appropriate  when 
quality  can  be  determined  from  a  comparison 
of  conforming  bids  on  the  basis  of  price 
alone.  The  low  price  technically  acceptable 
proposal  route  requires  more  stops  along  the 
way  and  is  appropriate  when  quality  can  be 
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determined  on  the  basis  of  low  price  but  only 
after  a  thorough  evaluation  and  discussion  of 
technical  proposals  to  ensure  they  meet  more 
complex  government  requirements.  The 
route  that  is  known  as  the  best  value 
approach  is  appropriate  when  quality  cannot 
be  determined  by  low  price  alone.  This  route 
is  perhaps  the  least  understood  because  it 
requires  considerable  subjective  judgment  to 
negotiate  the  curves  and  detours  along  the 
way.  Under  the  best  value  approach,  there  is 
no  easy  formula  for  measuring  the  value  of 
benefits  the  government  receives.  While 
justification  is  always  necessary  to  support 
payment  of  a  price  premium,  the  need  to 
quantify  the  value  of  non  cost  benefits  will 
differ  in  each  case  and  depend  largely  upon 
whether  it  makes  sense  to  do  so  to  help 
support  the  decision. 

While  the  best  value  approach  is  not  new, 
the  recent  reforms  in  federal  procurement 
require  executive  agencies  to,  "promote  best 
value  rather  than  simply  low  cost  in  selecting 
sources  for  supplies  or  services."66  To 
accomplish  this,  there  must  be  substantial 
efforts  to  educate  both  the  government 
acquisition  work  force  and  industry  in  best 
practices  so  as  to  establish  a  common 
understanding  of  the  concept  and 
consistency  in  its  application  to  making  best 
value  business  decisions. 

The  ultimate  challenge  in  promoting  best 
value  will  be  to  develop  the  necessary  road 
maps  and  tools  to  incentivize  the  acquisition 
work  force  to  effectively  manage  rather  than 
avoid  risk  and  use  common  sense  and 
innovation  in  selecting  contract  sources 
which  provide  the  best  value  for  the 
American  taxpayer. 
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ABSTRACT 

This  paper  examines  key  microeconomic 
aspects  of  the  former  Soviet  Union  (FSU) 
military  systems  acquisition  process. 
Differences  in  the  FSU  and  U.  S.  military 
acquisition  processes  are  provided  using  a 
common  microeconomic  framework. 
Recommended  changes  to  U.  S.  military 
systems  acquisition  policy  are  provided 
based  upon  observations  of  FSU  policy  and 
hypothesized  outcomes  from  the 
microeconomic  framework. 

The  recommended  changes  to  U.  S.  policy 
involve:  (1)  selecting  initial  program 

performance  requirements  in  a  less  biased 
fashion  versus  the  anticipated  level  of 
program  cost  and  schedule,  (2)  set  the 
resulting  cost,  performance,  and  schedule 
(C,P,S)  solution  point  in  a  somewhat  more 
conservative  manner,  and  (3)  reducing  the 
degree  of  reluctance  to  decrease 
performance  during  the  course  of  the 
development  program  if  the  C,P,S  solution 
point  appears  to  be  infeasible. 

INTRODUCTION 

This  paper  examines  key  microeconomic 
aspects  of  the  former  Soviet  Union  (FSU) 
military  systems  acquisition  process.  A  brief 


discussion  is  given  of  the  FSU  military 
systems  acquisition  structure,  followed  by 
the  relevant  microeconomic  characteristics 
associated  with  the  FSU  counterparts  of  the 
U.  S.  military  system  buyer  (government) 
and  seller  (contractor).  Differences  in  the 
military  acquisition  processes  of  the  two 
countries  are  provided  using  a  common 
microeconomic  framework.  This  framework 
was  originally  developed  to  model  the  U.  S. 
military  acquisition  process.  Hypothesized 
outcomes  from  the  microeconomic 
framework  for  the  U.  S.  case  are  highly 
consistent  with  results  obtained  from  a  broad 
sample  of  U.  S.  defense  programs  spanning 
the  1950s  through  1980s  [1],  The  same 
preferences,  interactions  and  outcomes  also 
apply  to  highly  classified  ("Black")  programs 
and  NASA  space  programs  [1],  The 
microeconomic  framework  was  extended  to 
model  the  FSU  acquisition  process,  and  a 
summarized  version  of  the  framework  is 
presented  in  this  paper.  Finally, 
recommended  changes  to  U.  S.  military 
systems  acquisition  policy  are  provided 
based  upon  observations  of  FSU  policy  and 
hypothesized  outcomes  from  the 
microeconomic  framework. 

Substantial  differences  exist  between  the 
acquisition  processes  of  the  two  countries’, 
in  terms  of  preferences  between  buyers  and 
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sellers,  program  dynamics,  and  likely 
solution  points.  In  some  but  not  all  respects, 
the  FSU  acquisition  process  was  superior 
from  a  microeconomic  perspective  to  that 
commonly  employed  in  the  U.  S.,  where 
moderate  cost  and  schedule  growth  occurs  in 
most  major  military  acquisition  projects  [1], 

Considerable  change  has  existed  in  the  FSU 
and  its  military  acquisition  process  during  the 
past  five  to  10  years.  As  a  result,  a  high 
degree  of  uncertainty  surrounds  western 
analysis  of  the  FSU  acquisition  process.  It  is 
not  my  intent  to  address  the  highly  fluid 
nature  of  the  relatively  recent  FSU 
acquisition  process,  but  to  examine  in  some 
detail  the  process  in  place  until  about  1987. 
Since  some  evidence  suggests  that  the  Soviet 
military  acquisition  process  was  already 
changing  in  the  late  Brezhnev  period,  the 
description  and  model  presented  below  are 
most  applicable  to  the  “classical”  Soviet 
military  acquisition  process  from  roughly 
1950-1980  [2],  Although  this  may  appear 
dated,  the  majority  of  the  Cold  War  arms 
buildup  occurred  in  both  the  U.  S.  and  FSU 
prior  to  1987,  and  during  1950-1980.  Thus, 
a  comparative  analysis  through  1987,  with 
emphasis  from  1950-1980,  will  present  some 
important  lessons  learned  both  from  purely  a 
historical  perspective,  as  well  as  to  provide 
valuable  insight  for  future  U.  S.  military 
development  programs. 

(For  simplicity  sake,  I  will  generally  use  the 
term  Soviet  rather  than  FSU  throughout  this 
paper  except  in  a  few  specific  cases.  In 
addition,  I  use  the  U.  S.  military  acquisition 
program  phase  names  that  existed  between 
March  19,  1980  and  February  22,  1991, 
which  are  in  order  of  increasing  maturity: 
(1)  Concept  Exploration,  (2)  Demonstration 
and  Validation,  (3)  Full  Scale  Development, 
and  (4)  Production.  These  acquisition 
phases  correspond  to  those  currently  in  use, 
which  are  in  order  of  increasing  maturity: 


(1)  Concept  Exploration  and  Definition,  (2) 
Demonstration  and  Validation,  (3) 
Engineering  and  Manufacturing 

Development,  and  (4) 

Production/Deployment.) 

FORMER  SOVIET  UNION  MILITARY 
SYSTEMS  ACQUISITION  APPROACH 

An  early  typical  milestone  in  the  Soviet 
military  acquisition  process  was  when  the 
Soviet  service  armaments  directorate 
converted  a  military  systems  proposal  to  a 
draft  Tactical  Technical  Instruction  (TTI), 
which  specified  a  new  system's  rationale, 
estimated  cost,  and  operational  role.  (The 
information  in  this  section  was  extracted 
from  the  unclassified  work  of  Berenson  and 
Carley  [3],  the  Government  Accounting 
Office  [4],  and  Alexander  [5]  through  [11].) 
It  was  similar  to  the  Required  Operational 
Capability  document  in  the  U.  S.  When 
approved  by  the  government  and  military 
elements,  a  scientific-technical  commission 
reviewed  it  for  feasibility  and  producibility, 
possibilities  for  standardization,  etc.  (In 
effect,  the  estimated  cost  was  established  by 
the  system  requirements  negotiated  within 
the  military  and  the  subsequent  design.  On 
the  surface  it  was  very  similar  to  the  U.  S. 
case,  but  the  Soviet  development  phase 
starting  point  was  likely  more  conservatively 
set  because  of  risk  aversion  considerations 
than  in  the  U.  S.  case.)  The  proposed  Soviet 
military  system  was  then  examined  by  a 
research  institute,  which  performed 
exploratory  analyses.  The  resulting 
documentation  was  then  turned  over  to 
several  design  bureaus  to  lay  out,  among 
other  things,  their  own  system  concept.  The 
scientific-technical  commission  then  selected 
the  most  promising  design  or  designs,  acting 
as  a  source  selection  board. 
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The  next  step  was  to  convert  the  designers' 
concepts  into  semi-detailed  drawings  and 
then  into  working  prototypes.  These 
prototypes  sometimes  embodied  different 
technological  approaches  and  competed 
against  one  another,  especially  in  aircraft 
programs.  Prototype  design  and 

construction  methods  were  regulated  by 
design  handbooks  in  order  to  minimize 
program  risk.  Military  system  production 
did  not  occur  until  the  prototype  was 
proven.  Military  system  designers  were 
required  by  law  to  evaluate  alternative 
system  concepts  against  such  criteria  as 
reliability,  producibility,  and  standardization. 

Prototypes  underwent  extensive  laboratory 
and  factory  trials  conducted  by  the  design 
team,  factory  management  and  the  military 
customer  to  check  adherence  to  the  TTI. 
The  prototype  was  sometimes  refined  before 
"state  trials"  began.  (These  "state  trials" 
were  conducted  by  the  defense  ministry 
concerned,  the  Military  Industrial 

Commission  (VPK),  and  the  military 

commander.) 

Once  the  prototype  was  approved,  funds 
were  provided  for  the  equivalent  of  the  U.  S. 
Full  Scale  Development  (FSD)  program 
phase.  The  major  development  decision 
made  in  the  U.  S.  to  approve  the  resources 
required  to  proceed  with  FSD  is  the 

Milestone  II.  In  the  Soviet  Union,  it  was  the 
Joint  Decree  of  the  Central  Committee  of  the 
Communist  Party  and  the  Council  of 
Ministers  (responsible  for  implementing  the 
policies  originating  in  the  Communist  Party). 
The  Joint  Decree  was  the  first  and  possibly 
only,  major  approval  of  the  Soviet  Union 
whereas  it  is  the  second  of  the  three  major 
approvals  in  the  U.  S.  systems  acquisition 
procedure  (the  first  and  third  being 

Milestone  I  and  Milestone  III,  corresponding 
to  decisions  to  enter  the  Demonstration  and 


Validation  and  Production  phases, 
respectively). 

Technology  freeze  occurred  in  the  Soviet 
programs  about  the  time  the  Experimental 
Design  Work  (OKR)  Development  Phase 
(equivalent  to  FSD)  began;  while  it  is  usually 
years  later  in  the  middle  of  FSD  in  U.  S. 
programs,  as  a  result  of  the  Soviet  schedule 
and  U.  S.  performance  priorities. 

The  Soviet  design  team  was  not  finished  with 
the  military  system  when  it  was  turned  over 
to  the  production  plant.  A  design  team 
engineer  accompanied  the  transfer  and  was 
on  hand  until  production  was  completed. 
The  designer  had  a  large  say  in  the  military 
system  production  process,  and  was  held 
chiefly  responsible  for  quality  and  schedule. 

In  the  U.  S.,  accountability  for  system  design 
is  diffused.  Historically,  there  has  been  a 
strong  incentive  for  both  the  government  and 
contractors  to  make  overly  optimistic 
development  phase  proposals.  On  the  other 
hand,  “the  contractual  penalties  for  having 
been  overoptimistic  are  generally  small.” 
[12] 

However,  in  the  Soviet  case  the  designer  was 
prominently  identified  and  held  responsible 
for  success  or  failure,  which  contributed  to  a 
conservative  design  policy.  The  net  result 
was  to  drive  many  Soviet  military  system 
designs  to  incremental  levels  of  technology 
advance,  while  firmly  attempting  to  meet 
program  schedule  (and  to  a  somewhat  lesser 
extent  cost).  The  lack  of  severe  penalties  in 
the  U.  S.  case  removes  one  disincentive  for 
the  program  manager  and  contractor 
counterpart  in  attempting  to  seek  a  higher 
level  of  technology.  In  addition,  it  may  also 
indirectly  contribute  to  a  higher  level  of  U.  S. 
military  program  operating  slack  (which  may 
translate  into  increased  cost  and/or 
schedule). 
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U.  S.  resource  commitments  for  military 
programs  are  revisited  on  a  yearly  basis 
because  of  Congressional  approval 
requirements,  while  Soviet  resources  were 
typically  committed  for  the  full  program 
once  directed  by  the  issuance  of  a  Joint 
Decree. 

The  Soviet  military  did  not  have  a 
production  phase  budget  with  line  items  for 
individual  systems.  Instead,  once  the 
program  was  approved  through  a  Joint 
Decree,  the  Ministry  of  Finance  set  up  an 
account  for  the  designated  suppliers.  In 
effect,  there  were  no  financial  links  between 
the  military  (user),  the  Ministry  of  Finance 
(funding  source),  and  the  suppliers  (seller). 
Since  production  phase  funding  was 
contained  outside  the  Soviet  military  budget, 
the  budget  was  limited  to  personnel  and 
Operations  and  Support  (O&S) 
expenditures.  The  Soviet  military  budget 
was  different  than  the  U.  S.  military  budget, 
which  also  included  funding  for  the  system 
development  and  production  program 
phases.  (Historically,  financial  budgets  were 
not  a  principal  source  of  control  over  Soviet 
military  expenditures  but  rather  were  an 
after-the-fact  accounting  device  that 
followed  the  planning  of  physical  programs 
and  material  balances.) 

No  military  system  production  in  Soviet 
plants  began  until  development  and  testing 
was  complete.  Concurrency  in  Soviet 
programs  was  thus  rare,  except  for  crash 
programs.  Cost  overruns  were  not  tolerated 
historically  by  the  hierarchy,  and  cost 
estimates  included  padding  to  guard  against 
them.  (The  Soviet  policy  pertaining  to  cost 
overruns  was  apparently  softened  during  the 
past  decade.)  In  the  U.  S.,  concurrency 
occurs  in  some  military  programs;  cost  and 
schedule  estimates  tend  to  be  optimistic;  and 
cost  overruns  and  schedule  slippage  are 
common.  U.  S.  military  system  development 


phase  cost  and  schedule  growth  are  linked  to 
the  desire  to  meet  performance  requirements, 
coupled  in  some  cases  with  pushing 
technology  and  trying  to  manufacture 
unproven  designs. 

In  the  Soviet  military  system  procurement 
case,  the  buyer  equaled  the  seller  to  a  much 
greater  extent  than  in  the  U.  S.  case.  Hence, 
there  was  a  greater  tendency  to  "conform"  in 
the  Soviet  case  to  the  buyer's  wishes 
(particularly  if  pressure  was  applied  from  the 
central  government).  The  pressure  to 
conform  likely  existed  whether  or  not 
development  and/or  production  competition 
was  used. 

SOVIET  AND  U.  S.  MILITARY 
SYSTEMS  ACQUISITION  PRIORITIES 

Historically,  the  top  priority  of  Soviet 
programs  was  to  meet  a  preset  schedule; 
while  the  top  priority  of  U.  S.  programs  is 
performance.  In  addition,  the  U.  S.  tends  to 
use  new  technology  for  each  new  military 
system  to  provide  maximum  performance 
and  to  serve  as  a  force  multiplier,  albeit  that 
increasing  performance  for  a  given  level  of 
technology  will  tend  to  increase  cost  and/or 
schedule.  The  Soviets  historically 
emphasized  the  extensive  use  of  technology 
transfer  to  keep  pace  with  the  technology  in 
threat  impacted  systems,  and  used  off-the- 
shelf  components,  with  modest  performance 
upgrades  in  some  cases,  for  producibility 
considerations  to  help  meet  the  schedule  in 
other  programs.  Designs  were  typically  well 
within  the  world-level  state  of  the  art,  and 
performance  goals  were  modest  for  all  but 
stressing  systems  (e.g.,  threat  impacted).  In 
addition,  the  Soviets  rarely  included  more 
than  one  new  technology  at  a  time  in  a 
military  system  design.  However,  the 
Soviets  could  and  did  develop 
technologically  advanced  systems  when 
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needed  (e.g.,  Typhoon  submarine).  The 
Soviets  also  emphasized  relatively  low  cost 
military  systems,  albeit  at  somewhat  lower 
levels  of  performance  than  comparable  U.  S. 
systems,  and  compensated  for  the 
performance  difference  by  the  procurement 
of  large  quantities. 

There  was  greater  use  of  product 
improvements  in  the  Soviet  Union,  which 
proceeded  in  parallel  with  new 
developments.  The  use  of  product 
improvement  reduced  the  need  for  the 
Soviets  to  push  technology  in  each 
development  program  since  if  the  next 
program  didn't  achieve  the  desired 
performance  level,  the  one  starting  a  few 
years  later  would  come  closer.  In  addition, 
the  phasing  of  product  improvements  and 


new  system  developments  allowed  full 
employment  to  be  maintained  and  effectively 
used  at  the  major  design  bureaus. 

Given  these  constraints,  and  the  Soviet 
preference  for  low  cost  designs,  simplicity, 
standardization,  and  incremental 
improvement  were  often  evident  in  Soviet 
military  programs.  A  summary  of  some  of 
the  differences  in  U.  S.  and  Soviet  Design 
Practices  is  given  in  Table  1  [13], 

The  Soviet  design  philosophy  tended  to 
minimize  uncertainty  in  the  system 
development  phase,  and  reduced  the  number 
of  problems  in  the  production  phase.  The 
Soviet  acquisition  process  was  a  sequence  of 
disciplined,  risk-avoiding  steps  (except  for 
specially  designated  higher  risk  military 


SOME  UNITED  STATES  AND  FORMER  SOVIET  UNION 
MILITARY  DESIGN  PRACTICES 

TABLE  1 


United  States 

Element  of  Design  Design  Practice 


Soviet  Design  Practice 


Design  philosophy  Design  to  maximize 

performance 

Growth  philosophy  Advanced  technologies, 

Emphasis  on  growth 
potential 


Design  to  functional 
requirements 
Low  risk  design, 

Little  growth  potential 


Reliability 

Modification 


Designed-in 

Minor;  new  hardware  to 
meet  new  requirements 


Off-the-shelf  components; 
proven  technology 
Extensive  modification 
programs 


systems).  Tight  schedule  deadlines  were 
typically  imposed,  and  approval  of  each  step 
completed  was  marked  by  the  joint 
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signatures  of  system  designers,  government 
monitors,  and  military  customers.  Design 


agreements  were  legally  binding  and 
accountability  was  clear. 

As  a  result  of  these  considerations,  major 
Soviet  military  development  programs 
tended  to  be  much  more  stable  than  U.  S. 
military  development  programs  (everything 
else  held  constant). 

However,  advances  in  electronics, 
computerization,  information,  and 
miniaturization  since  the  1970s  left  the 
Soviet  military  industry  in  a  vulnerable 
position.  The  speed  of  change,  the 
complexities  of  design,  the  integration  of 
many  different  technologies  and  inputs  from 
diverse  sources,  and  the  high  precision  and 
reliability  needed  in  production  struck  at 
traditional  weak  points  of  the  Soviet 
economy,  which  adversely  affected  Soviet 
military  systems  acquisition. 

In  the  Soviet  case,  the  central  government 
dictated  military  system  cost  and  schedule  to 
a  moderate  extent  (more  so  than 
performance).  In  addition,  the  buyer  and 
seller,  reporting  to  the  central  government, 
operated  in  an  environment  that  "suggested" 
that  they  conform  in  part  to  the  wishes  of  the 
central  government.  This  was  another  force 
at  work  that  led  to  incremental  performance 
improvements  unless  the  central  government 
approved  a  program  with  considerably 
enhanced  technology  and/or  performance.  It 
reduced  the  absolute  level  of  Soviet  versus 
U.  S.  system  performance  in  some  cases,  but 
then  it  also  reduced  the  associated 
cost:  performance  (C:P)  and 

schedule:performance  (S:P)  related  program 
risk  as  well. 


SOVIET  AND  U.  S.  ACQUISITION 

MICROECONOMIC 

CHARACTERISTICS 

The  key  parties  in  Soviet  military  system 
development,  production,  and  O&S  phases 
were  the  Design  Bureaus  (development 
phase)  and  the  relevant  military  service  (user, 
O&S  phase).  The  Design  Bureau  was 
analogous  to  U.  S.  defense  contractors  for 
the  development  phase,  and  the  Soviet 
military  service  was  equivalent  to  its  U.  S. 
counterparts  for  the  O&S  phase.  As 
previously  mentioned,  there  were  no  direct 
links  between  the  military  service,  Ministry 
of  Finance,  and  suppliers  involved  in  the 
production  phase.  Thus,  the  relevant  parties 
for  microeconomic  purposes  in  Soviet 
military  systems  acquisition  were  the  Design 
Bureaus  and  the  military  service  (In  the 
military  procurement  case  the  buyer  and  the 
user  can  be  treated  as  parties  with  identical 
preferences.  However,  this  was  not 
necessarily  the  case  for  civilian  items  where 
different  preferences  sometimes  existed.) 

On  the  surface  this  might  seem  to  lead  to  a 
disjointed  set  of  interactions  between  key 
parties.  However,  interactions  did  occur 
between  the  parties  both  directly  and  via  the 
VPK  during  the  entire  military  system 
lifetime.  Thus,  although  the  Design  Bureaus 
and  user  interrelated  in  a  somewhat  different 
fashion  and  contained  a  different  structure 
than  in  the  corresponding  U.  S.  case,  both 
the  Soviet  and  U.  S.  cases  can  be  treated  as 
bilateral  cases  between  relevant  parties 
representing  the  systems  acquisition  process. 

The  Design  Bureau  typically  desired  to 
increase  development  phase,  hence  life  cycle, 
cost  in  order  to  insure  full  employment. 
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(The  development  phase  cost  was  allowed  to 
grow  to  a  certain  degree  in  order  to  meet 
performance  and  schedule  requirements.) 
The  Design  Bureau  also  typically  desired  to 
decrease  performance  and  increase  schedule 
in  order  to  have  the  highest  probability  to 
meet  system  design  requirements.  In  effect, 
the  Design  Bureau  attempted  to  minimize  the 
risk  of  failure,  not  just  for  the  development 
phase,  but  for  the  total  system  life  cycle. 

Given  the  typically  greater  disincentives  for 
failure  in  the  Soviet  versus  U.  S.  systems, 
there  was  a  clear  desire  on  the  part  of  the 
Design  Bureau  to  deliver  what  it  promised. 
Pressure  on  the  Design  Bureau  contributed 
in  part  to  the  relatively  conservative  design 
philosophy  (compared  to  the  U.  S.)  except  in 
special  cases  where  high  technology  (and 
risk)  design  solutions  were  necessary  and 
authorized.  This  is  in  strong  contrast  to  the 
U.  S.  case  where,  as  stated  by  Charles  J. 
Hitch  and  Roland  N.  McKean  in  1960: 
"Excessive  optimism  in  drawing  up 
performance  specifications  can  make  the 
development  so  difficult  that  it  must  fail,  or 
take  much  longer  and  cost  much  more  than 
planned,  or  require  a  downgrading  of  the 
requirements.  It  is  not  unusual  for  weapon 
system  requirements  to  be  so  optimistic  that 
several  inventions  or  advances  in  the  state  of 
the  art  are  needed  on  schedule  if  the 
development  is  to  succeed."  [14] 

The  Soviet  military  service  (user)  typically 
desired  decreased  O&S  and  life  cycle  cost  (in 
order  to  obtain  a  greater  quantity  of  a  given 
system),  increased  performance,  and 
decreased  schedule  for  similar  reasons  as  the 
government  in  the  U.  S.  systems  acquisition 
case.  The  Soviet  user  also  desired  increased 
system  reliability,  and  thus  the  associated 
increase  in  performance  and  decrease  in 
O&S  cost  that  accompanied  it.  (The  Soviet 
user  generally  favored  a  trade  involving 


lower  operational  performance  for  increased 
mission  performance  (reliability).) 

In  acquiring  U.  S.  military  systems,  the 
government  generally  prefers  lower  over 
higher  cost  and  schedule,  and  higher 

performance,  while  the  contractor  prefers 

higher  cost,  performance,  and  schedule. 

Lower  costs  are  desirable  to  the  U.  S. 
government  to  develop  more  military 
systems  for  a  fixed  budget  (or  the  same 
number  for  a  reduced  expenditure).  Shorter 
schedules  that  enable  the  system  to  become 
operational  earlier  enhance  the  force 

structure  and  military  balance  of  power. 

Higher  performance  permits  increased 
operational  capability  for  the  mission. 

U.  S.  contractors  prefer  higher  costs  because 
they  increase  profits.  Longer  schedules  are 
also  desirable  to  maintain  a  stable  work  force 
and  a  long-term  working  relationship  with 
the  government,  which  gives  the  contractor  a 
potential  competitive  advantage  for  follow- 
on  or  future  contracts.  Contractors  prefer 
high  performance  to  improve  their  potential 
competitive  advantage  in  the  high  technology 
arena  that  often  provides  a  potential 
competitive  advantage. 

In  the  Soviet  case  the  user  generally  had  in 
mind  a  maximum  cost  and  schedule  and  a 
minimum  performance  level,  while  the 
Design  Bureau  had  a  maximum  performance 
level  and  a  minimum  cost  and  schedule 
length. 

In  the  U.  S.  case,  the  government  will 
typically  have  in  mind  a  maximum  program 
cost  and  schedule  length,  along  with  a 
minimum  performance  level.  The  contractor 
will  often  have  its  own  minimum  cost  and 
schedule  length. 
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Typical  Iso-Schedule  U.  S.  Government  and  Contractor 
and  Former  Soviet  Union  User  and  Design  Bureau 
Cost: Performance  Technical  Possibility  Curve 
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A  technical  possibility  surface  encompasses 
the  region  of  feasible  cost  (C),  performance 
(P),  and  schedule  (S)  solutions,  and  is  the 
technical  program  constraint.  Points  on  the 
technical  possibility  surface  or  any  two- 
dimensional  slice  of  the  surface  indicate 
efficient  (but  not  necessarily  optimal) 
solutions.  (This  is  given  by  points  B,  F,  and 
C  on  the  C:P  slice  of  the  technical  possibility 
surface  (possibility  curve)  in  Figure  1.) 
Points  lying  above  a  possibility  curve 
indicate  feasible,  but  inefficient  solutions. 
Those  points  below  a  possibility  curve 
indicate  an  infeasible  solution. 

In  Figure  1  for  the  C:P  case  (for  a  given 
schedule),  points  D  and  G  above  the  curve 
represent  an  inefficient  combination  of  cost 
and  performance,  since  the  system  could  be 
developed  at  the  same  cost  but  with  higher 
performance  by  moving  to  the  right  or  at  the 
same  level  of  performance  with  less  cost  by 
moving  down. 

Points  to  the  right  of  the  C:P  curve  (A  and  E 
in  Figure  1)  are  infeasible  for  a  given  set  of 
input  constraints  (e.g.,  manufacturing 
processes,  technology  level,  or  program 
structure)  for  a  given  schedule.  This  point 
will  only  become  feasible  with  a  shift  to  the 
right  of  the  entire  C:P  curve,  thus  requiring, 
for  example,  improved  yield  for  a  given 
manufacturing  process  if  cost  is  held 
constant.  In  effect,  solution  points  A  and  E 
correspond  to  an  inappropriate  schedule 
length  selected  for  the  specified  cost  and 
performance  levels. 

It  is  anticipated  that  the  last  few  percent  of 
the  maximum  performance  possible  will  lead 
to  increasingly  greater  program  cost  and 
schedule  length  (the  first  and  second 
derivatives  of  cost  with  respect  to 
performance  are  positive).  This  often  causes 
a  major  problem  in  U.  S.  military  systems 
acquisition  where  performance  near  or 


exceeding  the  current  state  of  the  art  is 
typically  specified  for  new  development 
programs,  yet  insufficient  cost  and/or 
schedule  is  typically  budgeted  to  achieve  the 
desired  level  of  performance. 

Very  large  increases  in  unit  cost  can  result  in 
some  instances  as  the  performance  state  of 
the  art  is  reached.  (This  corresponds  to  a 
solution  point  in  the  vertical  portion  of  the 
C:P  slice  of  the  technical  possibility  surface 
(e.g.,  point  B  or  above  in  Figure  1).)  Such 
problems  are  not  limited  to  military 
programs,  affecting  commercial  technologies 
and  programs  as  well.  An  exaggerated 
instance  of  cost  increase  for  an  extremely 
minor  performance  change  is  given  as  an 
illustration.  Here,  the  purchase  price  of  a 
relatively  high  performance  U.  S.  commercial 
1,389,580  detector  element  visible  charge 
coupled  device  (CCD)  focal  plane  array  was 
examined.  The  only  pricing  criteria  for  the 
CCD  is  the  number  of  defective  detector 
elements,  and  as  it  goes  from  20  to  zero  (for 
1,389,580  total  elements),  the  resulting  price 
increases  by  a  factor  of  twelve.  Thus,  the 
last  0.0014  percent  of  available  normalized 
performance  leads  to  a  twelve  times  increase 
in  cost,  or  92  percent  of  the  total  cost! 

Given  a  conservative  and  incremental 
acquisition  strategy  and  design  approach,  the 
technical  possibility  surface  was  typically 
better  known  to  a  Soviet  planner  than  to  a  U. 
S.  planner.  In  effect,  the  Soviet  user  and 
Design  Bureau  did  not  typically  push  the 
desired  level  of  performance  to  the  steep 
portion  of  the  C:P  and  S:P  slices  of  the 
technical  possibility  surface  as  was  more 
likely  in  the  U.  S.  case.  In  addition,  the 
Soviets  historically  achieved  a  relative 
degree  of  cost  containment  in  many  military 
systems  by  conservatively  selecting  the 
operating  requirements,  corresponding 
performance  levels  necessary  to  meet  these 
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requirements,  and  system  design 
characteristics. 

Although  it  is  not  my  intention  to  focus  on 
the  1980-1987  period  in  this  paper,  during 
this  time  the  prowess  of  the  Soviet  military 
acquisition  process  began  to  degrade  versus 
the  1950-1980  period  due  to  three  reasons 
[2,  15],  First,  the  Design  Bureaus  began  to 
realize  that  they  had  greater  strength  and 
faced  fewer  penalties  than  in  the  1950-1980 
period.  Consequently,  the  dominance  of  the 
military  service  (buyer)  began  to  fade  and  the 
bargaining  power  of  the  two  entities  began 
to  shift.  Second  and  similarly,  the  ability  of 
the  buyer  to  coordinate  and  motivate  the 
Design  Bureaus  in  a  manner  consistent  with 
their  desires  began  to  diminish.  These  two 
reasons  contributed  to  cost  growth, 
performance  degradation,  and/or  schedule 
slippage  in  some  military  systems  deployed. 
Third,  Soviet  technology  began  to  lag  what 
was  needed  to  support  increasing 
performance  requirements  in  a  variety  of 
Soviet  military  systems.  This  corresponds  to 
solution  points  higher  up  on  the  C:P  and  S:P 
slices  of  the  technical  possibility  surface, 
with  higher  potential  levels  of  cost,  schedule, 
and  associated  risk. 

DISCUSSION  AND  CONCLUSIONS 

Figure  1  illustrates  typical  Soviet  and  U.  S. 
infeasible  solution  points  and  responses  to 
these  solution  points  in  the  C:P  case.  Given 
the  more  conservative  and  incremental 
Soviet  acquisition  strategy  and  design 
approach,  infeasible  Soviet  solution  points 
were  likely  to  be  closer  to  the  C:P  slice  of 
the  technical  possibility  surface  than  U.  S. 
solution  points  for  a  given  level  of  cost.  In 
the  Soviet  and  U.  S.  cases,  the  response  to 
an  infeasible  solution  point  with  respect  to 
C:P  typically  ranged  from  maintaining 
performance  and  allowing  cost  to  grow 


(points  F  and  B)  and  maintaining  cost  and 
allowing  performance  to  be  reduced  (point 
C).  U.  S.  development  phase  C:P  trade 
preferences  usually  strongly  favor 
maintaining  performance  and  increasing  cost 
[1],  as  indicated  by  point  D  in  Figure  1 .  In 
the  Soviet  case,  a  more  balanced  trade 
between  performance  and  cost  typically 
existed  in  response  to  an  infeasible  solution 
point,  as  indicated  by  point  D  in  Figure  1. 

The  C:P  slice  of  the  technical  possibility 
surface  has  positive  first  and  second 
derivatives  of  cost  with  respect  to 
performance  (as  shown  in  Figure  1).  As  the 
desired  level  of  performance  increases,  the 
potential  level  of  cost  rapidly  increases  (for  a 
constant  schedule).  In  addition,  the 
technical  possibility  surface  is  typically 
unknown  during  the  course  of  the 
development  phase.  Consequently,  slight 
miscalculations  with  respect  to  performance 
on  the  part  of  U.  S.  military  system  designers 
on  average  leads  to  a  moderate  level  of 
development  phase  cost  growth  [1], 

The  level  of  U.  S.  development  phase  cost 
growth  will  likely  be  greater  than  in  the 
Soviet  case  because  U.  S.  C,P,S  solution 
points  typically  have  higher  levels  of 
performance  coupled  with  the  strong  desire 
on  the  part  of  U.  S.  military  system  designers 
to  maintain  performance  rather  than  trading 
cost,  performance,  and  schedule  in  an 
unbiased  fashion.  This  issue  was  first 
identified  by  Marshall  and  Meckling  in  1959 
[16],  and  is  still  applicable  today:  "Typically, 
in  weapons  development  great  emphasis  is 
placed  on  performance.  Most  new  weapons 
are  developed  around  specific  detailed 
performance  requirements  laid  down  by  the 
military— requirements  that  are  taken  very 
seriously.  The  penalties  incurred  by  the 
contractors  for  not  meeting  performance 
requirements  are  more  severe  than  for  failure 
to  meet  availability  schedules  or  failure  to 
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live  within  original  cost  estimates.  As  a 
result,  whenever  circumstances  dictate  a 
retreat  from  early  plans,  it  is  usually  the  costs 
and/or  availability  that  gives  ground." 

The  resulting  solution  region  for  historical 
Soviet  military  systems  may  be  somewhat 
more  stable  and  better  defined  (less 
uncertain)  than  for  U.  S.  systems.  This  is  in 
part  because  utility  curve  tangencies  are 
possible  in  the  typical  Soviet  C:S,  C:P,  and 
S:P  cases,  whereas  they  are  only  possible  in 
the  typical  U.  S.  C:S  case.  (The  direction  of 
increasing  utility  for  the  Soviet  user  and 
Design  Bureau  were  opposites  in  the  C:S, 
C:P,  and  S:P  cases,  whereas  this  only 
occurred  in  the  U.  S.  C:S  case.  This  criteria 
is  typically  necessary  to  lead  to  tangencies 
for  a  given  pair  of  goods  (e.g.,  C:P).  Only 
when  tangencies  are  present  can  a  pareto 
optimal  contract  curve  exist,  which  may  lead 
to  a  more  stable  solution  region  (all  else  held 
constant).  However,  the  exact  solution  point 
achieved  in  such  a  case  will  depend  upon  the 
negotiating  strengths  of  the  two  parties 
[17].) 

In  effect,  historical  Soviet  military  systems 
were  typically  not  driven  towards  an 
infeasible  solution  region  or  programmatic 
boundary  constraints  as  frequently  as  U.  S. 
programs.  In  the  U.  S.  case,  no  tangencies 
involving  C:P  and  S:P  will  typically  exist, 
and  no  forces  are  usually  present  to  restrain 
movement  of  the  corresponding  development 
phase  solution  point  towards  increasing 
performance.  This  is  because  both  U.  S. 
government  and  contractor  utility  increase 
with  increasing  levels  of  performance.  In 
addition,  the  resulting  U.  S.  system 
development  phase  solution  point  may  also 
be  less  efficient  in  a  microeconomic  sense 
than  in  the  historical  Soviet  case.  This  is  a 
result  of  preferences  on  the  part  of  U.  S. 
planners  which  is  often  towards  relatively 
high  performance  systems  and  meeting 


performance  requirements,  rather  than  on 
trading  C,P,S  in  a  less  biased  fashion.  Since 
C,P,S  cannot  be  traded  perfectly  in  the  short- 
run,  the  cost  and  schedule  levels  that  result 
from  transforming  an  infeasible  solution 
point  to  a  feasible  one  are  likely  greater  than 
the  levels  that  could  have  been  achieved  for  a 
given  level  of  performance  had  the  solution 
point  initially  been  feasible  (e.g.,  a  more 
realistic  budget  and  schedule). 

However,  for  more  complex  Soviet  military 
systems  that  encompass  advanced 
technology  and  enhanced  missions  the 
potential  advantage  over  the  U.  S.  solution 
points  in  terms  of  stability  and  uncertainty 
was  likely  much  smaller. 

Both  historically,  and  for  more  recent, 
complex  military  systems,  the  Soviet 
bargaining  approach  centers  around  the  S:P 
utility  and  S:P  slice  of  the  technical 
possibility  surface,  with  the  corresponding 
C:S  and  C:P  utility  and  slices  typically  less 
important.  The  S:P  variable  pair  was  favored 
because  the  decreasing  order  of  preference 
for  both  the  Soviet  user  and  Design  Bureaus 
in  typical  (and  non-leading  edge)  military 
systems  was  for  schedule,  performance,  and 
cost.  High  technology,  hence  high 
performance  (leading  edge)  Soviet  military 
systems  often  had  a  relaxed  schedule  in  order 
to  reduce  overall  program  risk.  In  these 
cases  the  decreasing  order  of  preference  for 
both  the  Soviet  user  and  Design  Bureaus  was 
for  performance,  schedule,  and  cost. 

However,  with  rather  conservative 
performance  goals  and  design  standards  used 
for  all  but  leading  edge  military  systems,  it 
was  likely  that  the  Soviets  were  able  to 
achieve  modest  performance  improvements 
in  succeeding  generations  of  systems,  but 
with  far  less  cost  and  schedule  growth  and 
variation  than  experienced  in  U.  S.  systems. 
This  acquisition  strategy  was  also 
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emphasized,  and  to  a  great  extent  enforced, 
by  the  risk  mitigation  strategies  of  both 
parties  and  the  historically  relatively  greater 
bargaining  strength  of  the  user  versus  the 
Design  Bureaus. 

I  will  now  briefly  discuss  some  lessons 
learned  from  the  Soviet  military  acquisition 
process  that  can  likely  be  applied  to  the  U.  S. 
process  to  reduce  cost  and  schedule  growth. 
First,  initial  Soviet  program  performance 
requirements  were  made  in  a  less  biased 
fashion  versus  the  anticipated  level  of 
program  cost  and  schedule  than  in  the  U.  S. 
Second,  the  resulting  Soviet  C,P,S  solution 
point  was  generally  more  conservatively  set 
than  in  the  U.  S.  case,  which  led  to  reduced 
program  risk  and  turbulence  during  the 
course  of  the  program  (everything  else  held 
constant).  Feasible  U.  S.  military  system 
acquisition  solution  points  can  likely  be 
achieved  with  moderately  lower  cost  and 
schedule  for  only  a  small  decrease  in 
performance,  given  the  typical  infeasible 
solution  point  that  initially  exists.  Third, 
there  was  likely  less  reluctance  in  the  Soviet 
case  to  decrease  performance  during  the 
course  of  the  development  program  if  the 
solution  point  appeared  to  be  infeasible  than 
in  the  U.  S.  case. 

It  is  relatively  easy  to  identify  the  cause  of 
performance-related  problems  in  the  U.  S. 
military  systems  acquisition  process,  and 
show  how  the  level  of  such  problems  was 
likely  far  less  in  the  Soviet  acquisition 
process  for  typical  programs.  However,  it 
will  be  much  more  difficult  to  create  a 
paradigm  shift  to  deal  with  performance- 
related  problems  in  the  U.  S.  case  due  to  the 
underlying  incentives  that  are  inherent  in  the 
institutional  structure. 


Both  Soviet  and  U.  S.  military  acquisition 
programs  have  C,P,S  risk  due  to  an 
incorrectly  specified  technical  possibility 
surface,  which  results  from  uncertainty  in 
specifying  the  surface  coupled  with 
institutional  biases  and  preferences. 
However,  in  the  U.  S.  case,  this  coupled  with 
the  performance-related  emphasis  of  both  the 
government  and  contractors  leads  to 
moderate  cost  and  schedule  growth  for 
typical  U.  S.  military  development  programs. 

The  performance-related  emphasis  of  the  U. 
S.  military  systems  acquisition  process  was 
identified  by  Charles  J.  Hitch  and  Roland  N. 
McKean  (1960,  p.  252)  in  1960:  "Excessive 
optimism  in  drawing  up  performance 
specifications  can  make  the  development  so 
difficult  that  it  must  fail,  or  take  much  longer 
and  cost  much  more  than  planned,  or  require 
a  downgrading  of  the  requirements.  It  is  not 
unusual  for  weapon  system  requirements  to 
be  so  optimistic  that  several  inventions  or 
advances  in  the  state  of  the  art  are  needed  on 
schedule  if  the  development  is  to  succeed." 

The  U.  S.  performance-related  bias  is 
compounded  by  government  and  contractor 
decision  makers  who  are  typically  "graded" 
much  more  strictly  on  whether  or  not 
program  performance  objectives  are  met 
versus  meeting  cost  and  schedule  objectives, 
as  first  identified  by  Marshall  and  Meckling 
in  1959  [16], 

Placing  equal  or  near  equal  emphasis  on 
meeting  program  cost  and  schedule,  as  well 
as  performance  objectives,  will  be  necessary 
in  order  to  eliminate  the  strongly  ingrained 
U.  S.  government  and  contractor  program 
management  bias  favoring  performance  that 
has  existed  for  at  least  the  last  35  years  in 
U.S.  military  systems.  No  appreciable 
change  in  U.  S.  program  outcomes  is  likely 
to  occur  without  a  re-oriented  C,P,S 
emphasis. 
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ABSTRACT 

The  Department  of  Defense  (DoD)  has  a 
unique  approach  to  managing  the 
performance  of  its  suppliers,  codified  in  a 
management  standard  entitled  the 
Cost/Schedule  Control  Systems  Criteria.  I 
will  explore  the  hypothesis  that  the  DoD 
approach  is  actually  a  quality  management 
model  that  fits  within  the  guidelines  of  the 
international  quality  standard  ISO  9000. 
This  has  interesting  implications  given  the 
current  emphasis  within  DoD  on  acquisition 
streamlining  and  use  of  commercial 
practices. 

INTRODUCTION 

The  Cost/Schedule  Control  Systems 
Criteria  and  ISO  9000 

The  Department  of  Defense  (DoD)  has  a 
unique  approach  to  managing  the 
performance  of  its  suppliers,  one  that  is 
gaining  increased  respect  inside  and  outside 
the  DoD  as  a  highly  useful  program 
management  tool.  The  DoD  approach  is 
codified  in  a  set  of  management  standards 
called  the  Cost/Schedule  Control  Systems 
Criteria  (C/S CSC).  I  hypothesize  that  the 
C/SCSC  are  in  fact  a  quality  management 
model  that  fit  the  international  quality 
standard  ISO  9000.  I  propose  that  the  DoD 
consider  seeking  ISO  9000  certification  for 
the  C/SCSC  or  some  sort  of  reciprocity 
agreement  on  use  of  the  C/SCSC  to  satisfy 
the  management  guidelines  of  ISO  9000. 


To  explore  my  hypothesis,  I  will  highlight 
the  key  precepts  of  the  C/SCSC,  the  key 
management  precepts  incorporated  in  ISO 
9000,  and  compare  and  contrast  the  two 
models.  This  analysis  will  make  the  case 
that  the  C/SCSC  are  actually  a  quality 
management  model.  I  will  conclude  by 
exploring  possible  approaches  the  DoD 
might  consider  to  integrate  the  C/SCSC 
with  the  ISO  9000  commercial  practice. 

ISO  9000  and  the  C/SCSC:  A 

Comparison 

DoD  has  recently  embraced  the 
international  quality  standard  ISO  9000.  In 
February  1994  the  DoD  authorized  the  use 
of  ISO  9000  in  contracts  for  new  programs 
(McGovern,  1994).  DoD  has  used  the 
C/SCSC  as  a  contractor  performance 
management  approach  since  1967.  Both 
ISO  9000  and  the  C/SCSC  are  standards 
for  running  a  business  well.  They  are  is 
designed  to  apply  to  any  product  or  service 
made  by  any  process  anywhere.  In  order 
to  be  this  generic,  both  refrain  from 
mandating  specific  methods  or  techniques, 
leaving  these  choices  up  to  the  business 
managers.  Both  ISO  9000  and  the  C/SCSC 
require  businesses  to  document  their  own 
management  procedures.  ISO  9000 
requires  a  business  to  manage  in 
accordance  with  its  own  documented 
procedures,  as  long  as  they  meet  the  basic 
requirements  of  the  standard  (Rabbitt  and 
Bergh,  1994,  pages  9-10).  The  C/SCSC 
require  a  "system  description"  that 
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describes  the  management  system  and 
explains  how  it  meets  the  standards.  ISO 
9000  requires  that  all  activities  affecting 
quality  must  planned,  controlled  and 
documented  The  main  requirement  of  the 
C/SCSC  is  to  build  a  "performance 
measurement  baseline",  which  is  simply  a 
formal  documentation  of  the  processes 
required  to  deliver  under  the  contract  in 
question.  Controlling  the  performance 
measurement  baseline  so  that  costs  and 
quality  remain  within  plan  is  also  a  key 
requirement  of  the  C/SCSC.  The  main 
difference  between  ISO  9000  and  other 
quality  schemes  like  TQM  is  that  a  firm 
can  register  to  ISO  9000  after  an  accredited 
third-party  registration  body  certifies  that 
the  firm's  quality  system  conforms  to  the 
standard  (Johnson,  1993).  This  registration 
has  important  implications  for  a  firm's 
competitiveness,  and  firms  are  registering 
with  ISO  9000  in  record  numbers  for  this 
reason.  Firms  doing  significant  enough 
business  with  the  DoD  to  require  the 
application  of  the  C/SCSC  to  their 
management  system  must  submit  to  an 
audit  by  DoD.  The  intent  of  the  audit  is  to 
certify  the  firm's  system  as  compliant  with 
the  standard.  With  the  C/SCSC,  as  with 
ISO  9000,  certification  has  implications  for 
the  firm's  competitiveness.  The  similarities 
just  shown  between  ISO  9000  and  the 
C/SCSC  are  striking. 

The  ISO  9000  series  consists  of  five 
core  standards:  three  quality  assurance 
models  for  specific  environment  and  two 
documents  that  give  general  guidelines.  A 
simple  way  to  capture  the  interrelationships 
of  the  documents  that  comprise  the  ISO 
9000  standard  is  to  depict  them  in  a 
diagram,  as  below  (Gower,  1993,  p.30): 


tlesign/°r  production  Model  for  final 

development/  and  insta||ation  inspection  and 

production/ 

installation  and 

servicing 


Figure  1 

Upon  closer  inspection  the  ISO  standard 
that  is  most  similar  to  the  C/SCSC  is  ISO 
9004,  which  gives  guidelines  for  quality 
management  and  quality  systems  (American 
Society  for  Quality  Control,  1987).  The 
following  is  a  breakdown  of  the  ISO  9004 
elements,  with  discussion  of  how  the 
C/SCSC  deal  with  the  same  element 
indicated  in  italics.  This  discussion  draws 
on  the  1987  American  National  Standard 
Q94,  which  is  the  American  version  of  ISO 
9004,  addressing  Quality  Management  and 
Quality  System  Elements. 

ISO  9000  states  that,  in  order  to  be 
successful,  a  company  must  offer  goods  or 
services  that  satisfy  customer  expectations 
at  a  cost  that  will  yield  a  profit  at 
competitive  prices  (American  Society  for 
Quality  Control,  1987,  p  1).  The  C/SCSC 
are  a  clear  definition  of  DoD  customer 
expectations,  and  one  of  their  key  purposes 
is  to  insure  that  DoD  obtains  a  service  or 
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product  at  as  competitive  a  price  as 
possible.  The  C/SCSC  normally  only 
apply  when  DoD  flexibly  prices  its 
contracts,  which  it  does  in  order  to  offer 
reasonable  profit  incentives  to  companies  in 
the  high  risk  environment  of  DoD 
contracts.  Again,  there  are  strong 
similarities  between  ISO  9004  and  the 
C/SCSC. 

The  ISO  9000  mentions  that  the  scope  and 
field  of  applicability  of  the  standard  depend 
on  factors  such  as  market  being  served, 
nature  of  product,  production  processes, 
and  consumer  needs  (American  Society  for 
Quality  Control,  1987,  p  2).  A  discussion 
of  the  DoD  market  and  nature  of  the 
product  DoD  buys  will  show  clearly  that 
the  C/SCSC  are  simply  standards  well 
adapted  for  the  DoD  market  and  nature  of 
product..  At  one  extreme  of  the  spectrum  of 
possible  markets  is  the  truly  competitive 
market.  In  a  truly  competitive  market, 
there  are  many  sellers  and  buyers  selling  a 
homogeneous  product,  so  the  invisible 
forces  of  the  market  set  the  best  price  and 
quality.  Customers  can  rely  on  the  market 
price  being  the  best  price  available  for  the 
best  quality  product.  They  should  not  need 
insight  into  a  suppliers'  internal 
performance  data  to  help  them  determine 
the  right  price.  The  suppliers  need 
performance  measurement  systems  in  order 
to  remain  competitive,  but  the  customer  can 
rely  on  market  price  as  an  indicator  of 
value.  The  less  competitive  a  market,  the 
less  the  customer  can  rely  on  the  invisible 
forces  of  the  market  to  set  a  good  price  for 
a  good  quality  product.  The  proliferation 
of  ISO  9000  registered  firms  is  a  sign  that 
customers  cannot  strictly  rely  on  market 
price,  but  are  looking  for  other  assurances 
of  product  quality  and  price.  DoD  often 
buys  in  a  non-competitive  market.  For 
instance,  DoD  buys  weapon  systems  still  in 


the  conceptual  phase  or  research  and 
development  phase,  where  there  is  no 
similar  product  in  existence  and  the 
technical  risk  is  so  high  that  it  is  difficult  to 
know  what  the  system  will  cost.  DoD  may 
be  the  only  buyer  of  such  a  system. 
Furthermore,  there  may  only  be  a  few  or 
even  just  one  seller.  In  such  a  market, 
there  are  no  invisible  market  forces  that  set 
price.  Instead,  the  contracting  parties 
negotiate  estimated  prices  and  resort  to 
using  flexibly  priced  contracts  to  share  the 
cost  risk.  In  a  flexibly  priced  contract,  the 
price  the  government  pays  will  vary 
depending  on  the  suppliers'  actual  costs  to 
perform.  This  contracting  practice  places 
much  cost  risk  on  the  government,  which  it 
attempts  to  mitigate  by  requiring  insight 
into  internal  contractor  performance 
management.  If  the  DoD  is  not  satisfied 
with  a  contractor's  progress,  it  can  take 
early  management  action  to  terminate  the 
contract  and  limit  its  losses.  DoD  has  been 
a  pioneer  in  the  area  of  contractor 
performance  measurement  via  standards, 
probably  because  of  the  high  risk  nature  of 
its  market.  The  DoD  has  been  using  the 
C/SCSC  since  1967,  whereas  the 

commercial  world  adopted  the  ISO  9000 

standard  in  1987  with  recent  growth  in 
number  of  registrations  fueled  by 

international  competitive  pressures  (Rabbitt 
and  Bergh,  1994,  p  9.).  The  C/SCSC  are 
standards  for  management  that  are 

particularly  well  suited  to  the  DoD  market 
and  nature  of  product  DoD  buys,  and  as 
such  the  C/SCSC  conform  to  the 

requirements  of  ISO  9004. 

Section  4  of  ISO  9004  addresses 
management  responsibility.  It  states  that 
the  responsibility  for  and  commitment  to  a 
quality  policy  belongs  to  the  highest  level 
of  management,  that  there  should  be  a 
documented  quality  policy  with  defined 
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objectives  pertaining  to  key  elements  of 
quality.  It  further  states  that  management 
should  develop  and  implement  a  quality 
system  that  insures  satisfaction  of  customer 
requirements  and  emphasizes  problem 
prevention  (American  Society  for  Quality 
Control,  1987,  pages  2-3).  The  C/SCSC 
require  involvement  at  the  highest  program 
management  levels,  a  documented  policy 
with  defined  objectives  such  as  variance 
thresholds  and  completion  of  the  contract 
on  time  and  within  negotiated  contract  cost. 
The  emphasis  is  on  problem  prevention 
early  through  a  good  management  control 
loop.  In  this  area,  again,  the  C/SCSC 
conform  to  the  ISO  9000. 

ISO  9004  requires  clear  identification  of 
general  and  specific  responsibilities  and 
authority.  The  organizational  structure  and 
lines  of  authority  should  be  clear 
(American  Society  for  Quality  Control, 
1987,  p  5)..  The  C/SCSC  subsection 
entitled  Organization  deals  with  the  same 
issues. 

ISO  9004  requires  management  to  provide 
sufficient  and  appropriate  resources  to 
achieve  quality  objectives  (American 
Society  for  Quality  Control,  1987,  p  5). 
The  C/SCSC  subsection  entitled  Planning 
and  Budgeting  deals  with  the  allocation  of 
resources  in  accordance  with  the 
company's  need  to  produce  the  good  or 
service  at  a  given  price  in  an  agreed  to 
period  of  time  and  meeting  the  quality 
specifications  of  the  contract. 

ISO  9004  requires  that  the  quality  system 
allow  adequate  and  continuous  control  over 
all  activities  affecting  quality.  The 
management  system  should  emphasize 
preventive  actions  and  allow  the  ability  to 
correct  failures  should  they  occur.  Written 
procedures  that  state  simply, 


unambiguously,  and  understandably  the 
methods  to  be  used  in  management  control 
should  exist  (American  Society  for  Quality 
Control,  1987,  p  5).  The  C/SCSC  focus  on 
control  of  processes  or  activities  necessary 
to  deliver  the  product  or  service.  The 
entire  concept  of  building  a  performance 
measurement  baseline,  which  is  at  the  heart 
of  the  Planning  and  Budgeting  Section  of 
the  C/SCSC,  has  to  do  with  setting  up  a 
baseline  for  management  control.  The 
Analysis  section  of  the  C/SCSC  then  deals 
with  the  actual  control  process,  namely  the 
identification  of  problems  and  the  need  to 
take  and  document  remedial  action.  Here 
again,  the  C/SCSC  and  the  ISO  9000  are  in 
agreement. 

The  ISO  9004  devotes  a  number  of 
paragraphs  to  the  need  to  document 
procedures.  This  too  is  an  integral  part  of 
the  C/SCSC.  A  C/SCSC  compliant 
management  system  must  have  a  written 
system  description  and  managers  at  key 
control  points  must  maintain  a  document 
flow.  Documentation  is  essential  for  proper 
audit  of  the  system,  which  ISO  9004  states 
is  an  essential  part  of  a  management 
system.  ISO  9004  requires  regular  internal 
audits  and  a  written  audit  plan  formulated 
by  company  management  (American 
Society  for  Quality  Control,  1987,  p  6). 
The  C/SCSC  require  regular  internal  audit, 
though  traditionally  this  has  been  done  by 
an  agency  outside  the  company,  namely  the 
cognizant  government  contract 
administration  office.  The  trend  in  DoD 
now  is  toward  internal  company  "self- 
surveillance.  " 

After  the  above  general  topics,  ISO  9004 
moves  to  more  specific  areas  such  as 
quality  in  marketing,  specification  and 
design,  procurement,  and  production.  The 
C/SCSC  are  not  this  specific  because  they 
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are  not  designed,  as  ISO  9004  is,  to  be 
integrated  with  another,  much  more 
specific  standard  like  ISO  9001,  9002,  or 
9003.  However,  the  issue  of  control  of 
nonconformity  and  corrective  action  is  an 
integral  part  of  all  the  ISO  9004  areas.  For 
instance,  ISO  9004  discusses  corrective 
action  in  detail.  It  states  that  it  is  necessary 
to  define  the  responsibility  and  authority 
for  instituting  corrective  action  and  that 
management  should  evaluate  the 
importance  of  problems,  investigate  causes, 
and  take  corrective  action  (American 
Society  for  Quality  Control,  1987,  pages  6- 
15).  This  is  a  key  element  of  the  C/S  CSC 
standard,  with  the  requirement  to  identify 
who  the  "cost  account  managers "  are.  The 
cost  account  managers  are  the  managers 
responsible  for  instituting  corrective  action. 

The  similarities  between  ISO  9004  and  the 
C/SCSC  are  remarkable.  To  make  them 
just  a  bit  clearer,  it  may  help  to  place  ISO 
9000  into  its  place  in  the  overall  quality 
spectrum  and  then  compare  how  the 
C/SCSC  fit  in  this  spectrum  as  well.  Many 
different  quality  experts  have  approached 
the  subject  from  slightly  different  angles, 
but  there  are  common  threads.  One  author 
has  distilled  the  quality  issue  down  to  seven 
critical  factors  for  business  success: 

1 .  Customer  focus 

2.  Process  versus  results  focus 

3.  Management  commitment/responsibility 

4.  Continuous  improvement 

5.  Less  than  20%  of  problems  caused  by 
workers 

6.  Performance  measures 

7.  Cross  functional  councils  create 
constancy 

This  author  contends  that  ISO  9000 
addresses  the  first  three  factors,  that  it  is  at 
the  very  basic  end  of  the  quality  evolution, 


with  Total  Quality  Control  and  Total 
Quality  Management  at  the  mature  end  of 
the  quality  evolution  (Rabbitt  and  Bergh, 
1994,  p  21-24).  I  would  add  to  this  that  I 
have  demonstrated  that  the  C/SCSC  address 
these  first  three,  as  well  as  focus  6.  The 
C/SCSC  specify  a  particular  performance 
measurement  technique  called  earned 
value.  A  comparison  of  the  C/SCSC  with 
the  Total  Quality  Management  (TQM)  that 
Deming  espoused  will  help  to  clarify  the 
position  of  the  C/SCSC  in  the  quality 
spectrum. 

TQM  and  C/SCSC  —A  Comparison 

Let's  start  with  the  top  level  desired 
outcome  or  strategic  objective  of  TQM  and 
the  C/SCSC.  The  desired  outcome  of 
TQM  is  a  quality  product  or  service 
produced  at  the  lowest  cost  and  meeting  the 
expectations  of  customers.  This  is  strictly 
in  agreement  with  the  ISO  9004,  which  we 
have  already  shown  the  C/SCSC  to 
conform  to  as  well.  Thus,  ISO  9000,  TQM 
and  C/SCSC  have  a  common  strategic 
objective,  as  you  would  expect  of  elements 
in  a  general  quality  spectrum  (Rabbitt  and 
Bergh,  1994,  p  24). 

In  order  to  achieve  their  objectives,  TQM 
and  C/SCSC  each  defines  a  strategy.  TQM 
embraces  a  philosophy  of  continual 
improvement  in  processes,  to  be  achieved 
through  a  Plan-Do-Check-Act  process. 
The  focus  on  continual  process 
improvement  is  not  an  implicit 
requirement  of  the  C/SCSC,  as  it  is  not 
with  ISO  9000.  Continuous  improvement 
is  a  mindset  that  is  at  the  higher  end  of  the 
quality  evolution  (Rabbitt  and  Bergh,  1994, 
p  24).  In  common  with  TQM,  however, 
the  focus  of  C/SCSC  is  on  baseline 
processes,  and  on  improvement  to  the 
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processes  in  order  to  keep  costs  under 
control.  To  clarify  the  commonality  of 
C/SCSC  and  TQM,  it  is  useful  to  break  the 
C/SCSC  and  the  TQM  down  as  a  Plan-Do- 
Check-Act  management  model  (Conway, 
1993).  This  analysis  will  show  that  the 
C/SCSC  are  a  formal  version  of  a  Plan- 
Do-Check-Act  TQM  management  process. 
The  C/SCSC  names  for  Plan-Do-Check- 
Act  are  Organization,  Planning  and 
Budgeting,  Accounting,  Analysis, 
Revisions. 

In  the  Plan  phase  management  identifies 
the  baseline  processes,  customer  needs,  and 
products.  With  a  baseline,  it  is  possible  to 
measure  improvements  in  existing 
processes  and  through  incremental 
improvements  succeed  at  delivering  a 
product  that  better  matches  customer  needs. 
Upon  close  examination  we  find  that  the 
Organization  and  Planning  and  Budgeting 
sections  of  the  C/SCSC  deal  precisely  with 
the  PLAN  step.  These  sections  address  the 
need  for  management  to  identify  processes 
that  result  in  products,  as  well  as  to  identify 
managers  responsible  for  the  products.  The 
products  are  defined  in  the  work 
breakdown  structure,  and  the  processes  are 
documented  in  a  performance  measurement 
baseline. 

The  Do  phase  is  the  phase  in  which 
management  takes  the  first  step  to  continual 
improvement  of  the  processes  documented 
in  the  plan  phase.  In  the  Do  phase,  the 
organization  puts  the  plan  into  action  and 
collects  data  on  the  activities.  In  the 
Check  phase,  management  analyzes  results 
from  the  Do  phase,  makes  comparisons 
with  expected  results,  and  attempts  to  gain 
a  clear  understanding  of  the  causes  of  the 
results.  This  is  also  the  phase  in  which 
management  identifies  process 
improvements.  In  order  to  make 


meaningful  continual  process 

improvements,  it  is  essential  to  measure 
performance  of  the  baseline  processes 
meaningfully.  If  the  measurement  metric  is 
not  a  meaningful  measure  of  performance, 
process  improvements  proposed  based  on 
analysis  of  that  metric  are  likely  to  be  off 
the  mark.  Performance  measurement 
metrics  are  the  subject  of  numerous  books. 
The  measurement  technique  often 
associated  with  TQM  is  statistical  process 
control,  but  there  are  many  different 
performance  techniques.  The  trick  is  to 
choose  the  most  appropriate  performance 
measurement  technique  to  measure  each 
process.  The  Act  phase  is  the  final  step  in 
the  cycle.  In  this  step,  the  organization 
institutionalizes  the  improvements 
identified  in  the  Check  phase 

The  Analysis  section  of  the  C/SCSC 
addresses  the  Do-Check-Act  phases  of  the 
TQM.  The  C/SCSC  require  rigorous 
assessment  of  reasons  for  deviation  from 
plan,  impacts  on  the  product,  and  methods 
for  recovery  or  process  improvement  if 
there  is  a  significant  deviation  from  plan. 
Just  as  with  TQM,  the  choice  of  the  right 
performance  measurement  technique  is 
crucial  to  the  success  of  the  management 
process.  In  the  case  of  the  C/SCSC,  there 
is  a  requirement  to  use  earned  value  or 
Budgeted  Cost  of  Work  Performed  as  a 
performance  measurement  technique.  This 
technique  is  particularly  well  suited  to  the 
DoD  market,  but  it  is  simply  a  performance 
measurement  technique. 

With  the  completion  of  the  Act  phase,  the 
management  cycle  starts  over.  The 
baseline  plan  now  incorporates  the  process 
improvements  identified  in  the  previous 
cycle,  and  any  future  improvements  will 
build  on  those  that  have  been 
institutionalized  (Conway,  1993). 
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The  foregoing  comparison  makes  the 
essential  link  between  C/SCSC  and  TQM 
clear.  C/SCSC  terminology  may  obscure 
the  link.  For  instance,  the  remaining 
sections  of  the  C/SCSC,  Accounting  and 
Revisions,  deal  mainly  with  the  discipline 
of  the  Plan-Do-Check-Act  or  performance 
measurement  system.  In  the  TQM  model, 
these  subjects  do  not  have  equal  standing 
with  Plan-Do-Check-Act,  which  are  the 
key  management  processes.  In  the 
C/SCSC  these  sections  receive  equal 
standing  with  the  key  management 
processes.  The  author  understands  that  the 
C/SCSC  joint  implementation  guide, 
currently  in  revision,  takes  a  management 
process  orientation. 

This  comparison  places  the  C/SCSC  into  its 
place  in  the  overall  quality  spectrum. 
Armed  with  the  foregoing  analysis,  it  is 
now  possible  to  consider  in  an  informed 
manner  how  the  DoD  might  integrate  the 
C/SCSC  with  the  ISO  9000  standard. 


Why  Not  Have  an  International 
Standard  for  C/SCSC  Similar  to  the 
International  Quality  Standard? 

The  foregoing  analysis  has  placed  the 
C/SCSC  in  its  place  in  the  quality  spectrum 
and  shown  the  similarity  between  the 
C/SCSC  and  ISO  9004,  the  management 
piece  of  the  ISO  9000.  The  DoD  has 
already  adopted  ISO  9000  overall  as  an 
acceptable  quality  management  standard.  If 
you  accept  the  argument  that  the  C/SCSC 
are  basically  a  DoD  version  of  ISO  9004, 
tailored  to  the  unique  needs  of  the  DoD 
market  and  product,  it  makes  sense  to  seek 
formal  recognition  of  this  link.  The 
defense  industry  is  familiar  with  the 
C/SCSC  approach,  ISO  9000  is  making 


growing  inroads  into  the  commercial 
world,  and  the  DoD  is  currently  focusing 
on  maximum  use  of  commercial  practices. 
There  are  many  approaches  the  DoD  might 
pursue,  but  one  that  comes  to  mind  from 
the  foregoing  analysis  is  to  seek  recognition 
of  the  C/SCSC  as  a  form  of  ISO  9004. 
This  would  allow  a  C/SCSC  certified 
company  to  use  the  C/SCSC  certification  as 
proof  of  an  acceptable  overall  management 
process,  with  the  applicable  ISO  9000 
standard  insuring  it  has  an  acceptable 
quality  process  in  place. 

CONCLUSION 

This  article  explored  the  hypothesis  that  the 
DoD  contractor  performance  management 
approach  embodied  in  the  C/SCSC  is  a 
quality  management  model  that  meets  the 
intent  of  ISO  9004.  The  methodology  of 
comparative  analysis  showed  the 
similarities  between  ISO  9000  and  the 
C/SCSC.  Considering  this  analysis,  the 
article  recommended  that  the  DoD  seek 
some  kind  of  recognition  that  the  C/SCSC 
are  ISO  9004  like  management  guidelines. 
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ABSTRACT 

In  1991  Defense  Management  Report 
Decision  Number  971  proposed  that  the 
Defense  Business  Operating  Fund  (DBOF) 
be  implemented  by  specific  Department  of 
Defense  (DOD)  activities.  The  initiative 
required  that  activities  identify  the  cost  of 
providing  services  to  their  customers  and 
charge  their  customers  for  those  services.  An 
operating  fund  would  be  created  to  provide 
financial  support  for  the  organization  at  the 
level  in  which  it  operated.  The  operating  fund 
concept  is  intended  to  impart  to  each  activity 
the  sense  of  operating  in  a  business  or 
competitive  environment.  The  lack  of  a 
competitive  environment  often  has  been  cited 
as  a  major  inhibitor  to  real  improvement  in 
performance  and  in  controlling  costs  in  the 
public  sector.  In  anticipation  of  implemen¬ 
tation  of  such  a  policy,  the  Contracts 
Department  at  the  Naval  Air  Warfare  Center 
Weapons  Division  (NAWCWPNS)  China 
Lake,  Calif.,  began  to  develop,  test,  and 
implement  the  unit  cost  concept. 

Within  NAWCWPNS  the  Contracts 
Department  has  been  operated  and  funded  as 
a  general  and  administrative  (G&A)  organiza¬ 
tion.  At  the  beginning  of  the  project,  the 
Contracts  Department  had  little  quantitative 
data  to  indicate  procurement  costs  to  process 
individual  requirement  actions,  total  output 
by  the  workforce,  or  effectiveness  of  the 
operations.  Department  personnel  were  not 
expected  to  perform  to  specific  quantitative 
standards  for  processing.  Consequently, 
procurement  officials  had  little  incentive  to 
streamline  processes  for  greater  efficiency. 
The  intention  of  the  Unit  Cost  Model  (UCM) 
project  was  to  provide  information  that  would 
provide  standards  and  measurement  capa¬ 
bility  to  support  process  improvements,  as 
well  as  fulfill  the  requirements  of  the  DBOF 
initiative. 

Early  in  the  development  phase  we  were 
introduced  to  the  Systems  Thinking  concept, 
which  encourages  viewing  a  system  as  a 


whole  to  understand  its  behavior  as  a  method 
of  creating  better  solutions.  Unwanted  conse¬ 
quences  can  be  predicted  through  model 
simulation  of  the  system.  Effective  solutions 
may  include  the  lessons  learned  through 
modeling.  We  used  Systems  Thinking  tools 
in  the  development  of  the  UCM  to  consider 
the  effects  of  various  choices  by  the  customer 
and  outside  activities  as  relates  to  the 
complexity  of  the  requirement.  The  predicted 
cost  and  direct  processing  time  to  execute  a 
procurement  action  are  based  on  the  total 
complexity  weighting. 

The  model  provides  the  customer  with  cost 
and  schedule  information  about  the  proposed 
procurement.  The  cost  estimate  will  vary 
depending  on  the  level  of  customer  service 
required,  and  the  amount  of  lead  time 
available.  The  model  also  provides  the 
Contracts  Deaprtment  with  standards  to 
measure  its  performance. 

This  paper  addresses  the  methodologies  used 
to  conceptualize  and  develop  the  UCM, 
implementation  of  the  model,  and  the 
resulting  changes  in  Procurement  Department 
operations. 

INTRODUCTION 

From  1985  to  1990  the  Procurement 
Department  at  China  Lake  went  through  a 
cultural  change  from  a  traditionally  managed 
procurement  shop — part  of  a  Naval  Supply 
Department — to  a  more  open  and  innovative 
culture  where  workers  were  empowered.  One 
method  of  empowering  workers  was  through 
their  participation  in  Procurement  Improve¬ 
ment  Action  Teams,  or  “Impact”  Teams. 
These  teams  were  chartered  by  the  Executive 
Steering  Group  of  the  Contracts  Department 
to  identify  systemic  problems  and  provide  the 
mechanisms  for  process  improvement. 
Among  those  teams  chartered  was  the 
Workload  Management  Impact  Team  (WIT), 
which  was  tasked  with  the  responsibility  of 
identifying  a  method  to  measure  workload. 
Specifically,  the  team  was  to  quantify  the 
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complexity  of  both  preaward  and  contract 
administration  work,  so  that  a  comprehensive 
and  fair  comparison  of  all  work  could  be 
made  beyond  simple  numbers  of  actions  and 
estimated  dollar  value  of  the  actions. 

The  systemic  problem  that  necessitated  the 
forming  of  this  team  was  the  consistent 
inability  of  department  managers  to  agree  on 
workload  and  allocation  of  resources  for 
adequate  support.  The  Contracts  Department 
was  composed  of  branches  that  supported 
finite  customer  groups.  Shifts  in  workload, 
due  to  changing  levels  of  support  required  for 
various  weapon  programs,  presented  a  tough 
choice  for  procurement  managers  regarding 
shift  in  personnel  or  reallocation  of  customers 
and  accompanying  workload  to  provide  an 
improved  level  of  customer  support. 

Before  the  creation  of  the  WIT,  management 
agreement  on  the  amount  of  work-in-process 
for  both  new  contract  actions  and  contract 
administration  was  difficult  to  achieve. 
Managers  with  a  high  count  in  terms  of 
actions  tended  to  refer  to  those  numbers, 
while  managers  who  provided  support  to 
programs  with  a  smaller  number  of  complex 
actions  generally  pointed  to  the  amount  of 
effort  required  for  support.  Managers  became 
aware  that  some  sort  of  consistent  method  to 
measure  work  must  be  identified.  As  far  as 
contract  administration,  managers  often 
described  the  workload  in  terms  of  “we  have 
a  lot.”  At  the  time  no  automated  system 
existed  to  provide  a  count  of  ongoing  contract 
administration  actions. 

The  advent  of  the  DBOF  initiative  during 
fiscal  year  1992  focused  our  concerns  about 
how  well  our  processes  really  worked,  given 
that  we  experienced  such  great  variation  in 
procurement  administrative  lead  time  (PALT) 
for  new  contract  actions.  At  the  time  we  were 
faced  with  an  increasing  widespread  geo¬ 
graphical  area  of  operations.  The  research, 
development,  test,  and  evaluation  facility  at 
China  Lake  had  consolidated  with  four  other 
sites  earlier  in  the  year,  with  the  new 
NAWCWPNS  spread  across  a  region  from 
New  Mexico  to  Hawaii.  As  a  result  of  these 
influences,  our  efforts  became  concentrated 
on  measurement  as  a  means  of  understanding 
our  costs  and  improving  our  processes.  As 


DBOF  specifically  required  that  an 
organization  know  its  output  of  products  and 
what  each  product  cost,  the  concept  of  a 
UCM  began  to  take  shape. 

INITIAL  TASK  PARAMETERS 

The  initial  concept  development  tasks  were 
direction  to  use  the  product  of  the  WIT, 
allocation  of  the  cost  over  the  PALT,  and  the 
concept  of  a  variable  rate  of  customer  service. 
The  resulting  model  was  to  have  a  three- 
dimensional  appearance,  with  the  three 
specified  areas  serving  as  axes. 

Can  Cost  Be  Measured  Per  Day? 

Because  I  had  served  on  the  WIT  and  played 
a  role  in  development  of  its  product,  I 
addressed  the  PALT  portion  of  the  model 
first.  After  several  days  of  calculating  the 
mean  and  standard  deviation  of  PALT  for  the 
awards  found  within  each  of  the  specified 
dollar  thresholds,  I  realized  that  allocating 
costs  based  on  the  number  of  days  required 
to  process  and  award  a  new  requirement 
could  not  be  accomplished.  Our  internal 
processes  were  so  out  of  control,  in  the  total 
quality  management  sense,  that  any 
determination  of  the  PALT  associated  with 
the  particular  feattires  of  the  requirement  had 
variances  up  to  50%  of  the  mean.  Any 
attempt  to  access  costs  on  this  basis  would  be 
grossly  unfair  to  the  individual  customer.  The 
other  reservation  I  had  about  accessing  costs 
on  the  basis  of  cost-per-day  of  lead  time 
incurred  was  that  this  concept  was  similar  to 
a  cost-as-a-percentage-of-cost  contract.  In 
theory  the  more  days  the  Procurement 
Department  spent  in  the  solicitation, 
negotiation,  and  awarding  of  a  requirement, 
the  greater  the  cost  to  the  customer.  This 
approach  would  actively  work  against  the 
goal  of  providing  incentives  for  process 
improvement.  The  only  prospect  of  coming 
up  with  a  credible  approach  was  to  develop 
an  hourly  rate,  and  estimate  the  number  of 
hours  required  in  the  processing  and 
execution  of  each  type  of  procurement  action. 
In  the  procurement  world  this  concept  would 
be  considered  an  engineering  estimate,  not 
generally  considered  the  preferred  method  for 
estimating  costs. 
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This  project  would  have  been  assisted 
immensely  by  the  existence  of  a  capable 
management  information  system,  but  only  a 
limited  system  was  in  place.  The 
Procurement  Department  at  NAWCWPNS, 
like  many  other  government  activities, 
primarily  measures  and  records  that 
information  required  for  reporting  outside  the 
department.  All  the  output  of  services 
provided  by  the  department  is  not  recorded  or 
known.  The  budget  requirements  for  a  G&A- 
funded  department  do  not  require  an  in-depth 
understanding  of  the  nature  of  expenses  and 
revenue. 

Customer  Service 

To  add  a  cost  for  customer  service,  the 
various  levels  of  customer  service  had  to  be 
defined.  In  attacking  the  problem,  I  divided 
customer  service  into  two  components.  From 
past  experience  as  a  contract  specialist,  I 
realized  that  part  of  the  customer  service 
provided  was  based  on  the  complexity  of  the 
requirement,  and  the  remainder  was  due  to 
the  preferences  and  needs  of  the  customer.  In 
keeping  with  the  overall  pricing  strategy,  the 
first  level  was  called  Basic  Processing,  which 
represents  a  zero  level  of  customer  service. 
This  level  was  offered  not  for  its  desirability 
but  rather  because  a  baseline  representing  no 
premium  cost  had  to  be  included  to  be 
comparable  with  other  organizations  that  had 
no  customer  focus.  Three  levels  were 
delineated  between  Basic  Processing  and  the 
highest  level,  Level  4  Service,  representing 
the  world-class  customer  service  we  often 
provide  to  major  program  offices.  Level  4 
Service  probably  would  be  indicated  by 
anticipation  of  travel  with  the  customer, 
support  for  program  office  briefings, 
meetings  with  sponsors,  and  perhaps  parti¬ 
cipation  on  a  team  with  the  customer  to  assist 
in  writing  the  acquisition  requirements 
package  before  it  is  brought  into  the 
procurement  office  for  processing. 

Customer  services  related  to  the  complexity 
of  the  requirement  are  at  the  following  levels 
and  definitions: 

•  commercial  or  part  numbered  item,  0% 
premium  effort 


•  fixed-price  noncommercial  supply 
requirement,  2%  premium  effort 

•  time  and  material  or  fixed-price 
services,  5%  premium  effort 

•  research,  development,  test,  and 
evaluation  requirement,  8%  premium  effort 

•  competitive  service  requirement  fea¬ 
turing  cost  realism  analysis,  10%  premium 
over  basic  processing. 

The  customer  service  costs  associated  with 
the  complexity  of  the  requirement  are  non- 
negotiable  cost  items.  The  desired  levels  are 
somewhat  at  the  discretion  of  the  customer. 
Even  though  five  levels  of  desired  customer 
service  are  featured,  we  recommend  that 
service  below  Level  2  will  most  likely  not 
meet  customer  needs.  Basic  Processing  and 
Level  1  Service  may  end  up  costing  the 
customer  more  in  time  for  evaluation, 
negotiation,  delays,  and/or  contract  adminis¬ 
tration  effort  than  the  higher  level  of  customer 
service.  Mid-level  service,  Level  2,  is  the 
minimum  level  of  customer  service  we 
provide  at  our  activity. 

Except  for  the  initial  level  denoting  no 
customer  service,  each  level  for  desired 
customer  service  is  identified  by  a  number. 
Briefing  contract  specialists  on  the  project 
and  identifying  the  levels  with  descriptive 
titles,  such  as  minimum  level  of  service,  high 
level  of  service,  brought  some  angry 
responses  by  individuals  refusing  to  consider 
downgrading  the  customer  service  provided 
from  the  world-class  level.  However, 
whether  we  choose  to  acknowledge  it  or  not, 
we  currently  informally  ration  the  customer 
service  we  provide,  based  on  priorities, 
visibility,  and  other  factors.  By  negotiating 
the  level  of  service  with  the  customer  up 
front,  we  can  establish  a  contract  where  the 
service  is  based  on  the  requirement  com¬ 
plexity  and  other  factors,  and  on  how  much 
the  customer  is  willing  to  pay.  Until  our 
department  culture  can  transfer  the  notion  of 
price  allocation  of  service,  the  assignment  of 
numbers  to  differentiate  between  the  levels  of 
desired  service  seems  more  acceptable  to  the 
workforce. 

Complexity  as  a  Price  Determinant 

Complexity,  as  determined  by  the  WIT,  is  the 
feature  that  provides  the  primary  deter- 
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mination  of  the  cost  to  the  customer.  Each 
requirement  is  assigned  a  complexity 
weighting  based  on  the  dollar  range  appli¬ 
cable  to  the  Government  estimate  of  the  cost 
of  the  requirement,  the  type  of  contract,  and 
other  features,  including  solicitation  by 
means  other  than  full  and  open  competition 
(OTF&OC),  using  best  value  method  of 
source  selection  or  planning  a  presolicitation/ 
preproposal  conference.  Weighting  for  the 
estimated  cost  of  the  procurement  action  is 
based  on  significant  acquisition  dollar 
thresholds  ($50K,  $500K,  $1M,  $10M,  and 
$50M)  where  the  value  of  the  procurement 
imposes  added  complexity  in  processing  of 
the  requirement.  In  selecting  attributes  for  the 
assignment  of  complexity  weighting,  the 
WIT  was  not  attempting  to  measure  the  effort 
for  the  individual  feature  but  the  complexity 
present  in  the  whole  process  when  the 
specific  feature  was  present.  For  example, 
having  a  preproposal  conference  may  not  take 
a  lot  of  effort  in  and  of  itself,  but  these 
conferences  are  not  generally  held  unless 
complicating  factors  are  present  in  the 
acquisition,  signaling  increased  efforts 
throughout  the  procurement  processing. 

After  the  complexity  weighting  is  determined, 
the  weighting  is  multiplied  by  the  number  of 
basic  hours  allocated  for  the  type  of 
instrument  being  processed.  An  adjustment  is 
made  for  any  split  or  other  award  made  from 
a  Broad  Agency  Announcement  where  only 
one  solicitation  is  written  or  an  oral 
solicitation  is  used  rather  than  the  normal 
written  solicitation.  The  product  is  the 
number  of  net  direct  hours  expected  to  be 
spent  directly  processing  the  requirement.  On 
top  of  the  direct  hours  is  added  a  factor  for 
time  spent  on  process  improvement,  time  for 
travel  in  support  of  technical  requirements  (an 
opportunity  cost  because  the  individual 
cannot  work  on  other  requirements  to 
generate  revenue),  and  a  factor  for  training. 

DOD  has  an  extensive  program  of  mandatory 
training  for  acquisition  personnel.  The 
number  of  changes  affecting  the  acquisition 
field  in  the  last  few  years  has  increased  the 
amount  of  refresher  training  employees  must 
engage  in  to  maintain  effective  skills  in  the 
profession.  The  training  received  typically 


provides  benefits  to  all  customers  not  just  a 
select  few.  For  this  reason,  a  factor  to  recoup 
the  costs  among  all  customers  for  time  spent 
receiving  training  is  a  necessary  part  of 
operations. 

The  final  factor  for  determination  of  the 
number  of  total  hours  required  is  the  amount 
of  customer  service  determined  by  procure¬ 
ment  and  technical  personnel  to  be  appro¬ 
priate  for  the  acquisition.  Multiplication  of  all 
the  indirect  factors  by  the  number  of  net 
direct  hours,  plus  the  number  of  net  direct 
hours,  equals  the  total  number  of  hours. 

The  average  hourly  rate  for  contract 
specialists  and  procurement  technicians  is 
applied  to  the  total  number  of  hours  for  direct 
personnel,  as  is  the  hourly  rate  for  all 
department  indirect  support  personnel  and 
non-labor  costs.  Indirect  labor  costs  include 
the  labor  for  contracting  officers,  clerical 
staff,  supervisors,  analysts,  administrative 
officers,  department  management,  and 
contract  management  advisors  (CMAs)  who 
serve  as  liaison  officials  in  support  of  the 
technical  customer.  Contracting  officers  were 
not  included  in  the  direct  labor  costs  because 
the  majority  of  their  time  is  spent  on  efforts 
of  general  support  benefiting  all  present  and 
future  work,  as  opposed  to  particular 
ongoing  actions.  CMAs  are  included  in 
indirect  labor  because  no  comparison  with 
other  contracting  activities  is  possible.  This 
function  generally  does  not  occur  within  most 
contracting  organizations  other  than  at 
NAWCWPNS.  The  direct  efforts  performed 
by  CMAs  occur  before  acceptance  and 
processing  by  the  Contracts  Department  and 
represent  only  a  portion  of  their  function, 
with  the  remaining  duties  clearly  representing 
indirect  activity.  Non-labor  costs  include 
budgeted  expenses  for  equipment  purchase, 
contracts,  travel  for  Contracts  Department 
personnel,  supplies,  utilities,  transfer  pay¬ 
ments  to  other  departments,  and  costs 
associated  with  purchases  and  depreciation  of 
federal  information  processing  (FIP)  equip¬ 
ment.  The  total  represents  the  estimate  of  all 
operating  costs  based  on  the  expected  share 
of  the  total  workload  represented  by  the 
requirement. 
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USING  PRICING  FEATURES  TO 
INFLUENCE  DEMAND  AND  OTHER 
PRICE  ADJUSTMENTS 

Customer  Discounts 

Customer  demand  for  government  procure¬ 
ment  services  typically  peaks  the  last  half  of 
the  fiscal  year,  primarily  because  of  the  long 
lead  time  associated  with  funding  of 
requirements.  The  imbalance  of  incoming 
workload  favors  longer  lead  times  in  the 
beginning  of  the  fiscal  year  and  mistakes  in 
the  latter  half  as  personnel  are  pressured  by 
too  many  demands. 

In  establishing  the  cost  structure  of  the  UCM, 
our  goal  was  to  establish  a  pricing  mechan¬ 
ism  within  the  NAWCWPNS  acqui-sition 
community  to  introduce  the  dynamics  of  a 
competitive  market.  This  policy  implemen¬ 
tation  is  in  stark  contrast  to  the  sole-source 
bureaucratic  environment  that  traditionally 
influenced  contracting  organizations  in 
inefficiently  implementing  processes.  We 
believed  that  the  best  way  to  implement 
process  improvements  to  cut  costs  and 
provide  better  service  to  our  customers  was 
to  train  ourselves  to  view  our  situation  as 
being  in  competition  for  customers  and  as  a 
business  base  within  a  market. 

We  recognized  that  as  consumers  we  react 
individually  to  price  incentives.  We  took  the 
concept  one  step  further  by  projecting  that 
individual  customers  or  program  offices 
would  attempt  to  maximize  their  budgets  by 
submitting  requirements  early  when  possible 
if  offered  financial  incentives  to  do  so.  In  this 
way  we  are  attempting  to  manage  demand  for 
our  services,  so  that  we  can  move  more  of 
the  requirement  entering  the  procurement 
service  queue  into  the  first  quarter  of  the 
fiscal  year.  During  the  first  quarter  of  the 
year,  the  department  experiences  the  most 
excess  capacity  in  resources.  Reducing 
excess  capacity  in  the  organization  in  the 
early  part  of  the  fiscal  year  promotes  effective 
operations  with  fewer  personnel.  Currently, 
the  Procurement  Department  must  be  staffed 
to  meet  the  workload  bulge  during  the  latter 
half  of  the  fiscal  year,  because  we  can’t  easily 
reduce  or  increase  our  workforce  according 
to  workload.  By  using  the  pricing  mechanism 


as  an  incentive  for  early  submission  of 
requirements  where  possible,  we  encourage  a 
greater  stabilization  of  workload  for  the  year. 

Premium  Pay 

When  our  customers  fail  to  meet  the 
prescribed  submission  dates  for  guaranteed 
award  in  the  same  fiscal  year  and  the 
customer  desires  the  requirement  be  awarded 
by  the  end  of  the  fiscal  year,  premium  pay 
will  be  charged  to  that  customer.  This 
concept  represents  an  important  difference 
between  the  number  of  days  required  to 
award  a  new  contract  and  the  number  of 
hours  required  to  process.  In  most  cases  our 
customers  do  not  realize  that  we  must 
perform  the  same  processing  for  actions 
whether  we  have  plenty  of  lead  time  or  not. 
To  eliminate  process  bottlenecks  the  contract 
specialist  often  must  hand  carry,  fax,  or 
otherwise  use  extraordinary  administrative 
procedures  to  influence  the  speed  at  which 
the  requirement  is  processed.  The  contract 
specialist  normally  can  work  on  other 
requirements  while  reviews  are  being  made. 
However,  when  the  available  lead  time  is 
insufficient,  the  contract  specialist  often  must 
accompany  the  requirement  while  reviews  are 
made.  This  represents  an  inefficient  use  of 
labor  resources  and  sometimes  requires  over¬ 
time  to  ensure  that  actions  are  accomplished 
by  the  deadline  quoted.  All  the  customer 
realizes  is  that  the  requirement  was  awarded 
in  a  shorter  amount  of  time  than  usual.  By 
charging  premium  pay,  the  customers  will 
begin  to  gain  a  better  understanding  of  the 
role  they  play  and  of  our  processes. 

Premium  pay  is  used  to  provide  incentive  to 
the  customer  to  submit  requirements  within 
the  published  lead  times  for  the  indicated 
dollar  thresholds  of  the  requirements. 
Charging  premium  pay  for  requirements 
submitted  late  in  the  fiscal  year,  those 
normally  accepted  on  a  "best  efforts"  basis, 
also  educates  the  customer  that  these  require¬ 
ments  may  be  awarded  with  less  PALT  days. 
But  the  number  of  hours  required  for 
processing  is  often  greater  because  a  number 
of  processes  require  hands-on  processing  to 
work  through  organizational  bottlenecks,  for 
example,  walking  documentation  through  for 
approval  to  obtain  legal  reviews,  review  by 
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senior  procurement  officials  before 
negotiation  and  award,  and  faxes  sent  to 
contractors. 

Service  Rebates 

Charging  a  premium  rate  for  failing  to  submit 
a  requirement  within  administrative  deadlines 
that  represent  lengthy  processes  outside 
statistical  process  controls  is  not  entirely  fair 
to  the  customer.  In  many  cases  at  the  point  in 
the  program  when  the  need  is  realized,  the 
customer  is  already  beyond  the  submission 
deadline  necessary  to  obtain  delivery  without 
experiencing  program  delays.  We  recognize 
that  procurement  officials  must  work  in 
partnership  with  their  customers  and  accept 
part  of  the  risk  in  acquisition.  Therefore, 
providing  a  rebate  to  the  customer  would  be 
appropriate  only  when  the  upper  range  of  the 
lead  time  is  exceeded  by  the  Contracts 
Department.  If  award  is  not  made  timely,  the 
program  manager  experiences  greater 
program  risk,  but  the  Contracts  Department 
also  experiences  an  operating  loss  on  the 
contract  action  and  increases  its  overall 
financial  risk. 

INFORMATION  FROM  MODEL 
OUTPUT 

When  a  customer  provides  the  Procurement 
Department  with  information  describing  a 
proposed  acquisition,  what  type  of  infor¬ 
mation  might  that  customer  expect  to  receive 
from  entry  of  descriptive  data  into  the  UCM? 
The  major  information  available  for  a  new 
contract  action  is  as  follows: 

Total  Adjusted  Preaward  Contract  Cost: 
Includes  all  labor  and  non-labor  costs,  and 
adjustments  for  required  award  date.  If  a 
rebate  or  discount  to  the  customer  is 
included,  this  cost  is  not  equivalent  to  the 
total  cost  to  process.  This  cost  represents  the 
cost  charged  to  the  customer. 

Contract  Complexity  Weighting:  Allows  a 
comparison  of  complexity  of  customer’s 
requirement  in  relation  to  other  requirements. 

Total  Hours:  Number  of  hours  of  direct 
personnel  attributed  to  processing  the  defined 
requirement,  plus  other  indirect  hours.  Also 


represents  the  number  of  hours  used  as  a 
multiplier  against  the  indirect  labor  and  non¬ 
labor  hourly  rates  to  determine  their 
contributions  to  the  total  cost. 

Basic  Hours:  Base  number  of  hours  for  the 
type  of  contractual  instrument  required  or 
planned. 

Predicted  PALT  and  the  Low  and  Upper 
Range:  Provides  the  customer  with  the 
predicted  administrative  lead  time,  and  the 
upper  and  lower  range  representing  best-case 
and  worst-case  scenarios  based  on  the 
complexity  characteristics  of  the  requirement. 

Total  Fixed  and  Variable  Costs:  Subsets  of 
the  total  adjusted  preaward  contract  cost. 

Total  Adjusted  Direct  Labor,  Indirect  Labor, 
and  Non-Labor  Costs:  Subsets  of  the  total 
adjusted  preaward  contract  cost. 

Net  Cost  For  Required  Award  Date 
Adjustment:  Adjustment  for  premium  pay  or 
a  discount  to  the  customer,  reflected  as  a 
negative  number,  for  early  submission  of  the 
requirement.  Included  in  the  total  adjusted 
preaward  contract  cost. 

Cost  of  Customer  Service:  Customer  can  see 
the  cost  of  customer  service  as  they  have 
defined  it,  allowing  variation  in  the  level  to  be 
entered  to  ultimately  determine  the  best  trade¬ 
off  of  service  to  meet  budget  and  program 
needs.  A  subset  of  the  total  adjusted 
preaward  contract  cost. 

Net  Direct  Hours:  Estimated  number  of  direct 
processing  hours  required  to  provide  the 
described  service. 

Total  Contract  Instrument  Award  Cost:  Cost 
exclusive  of  any  rebates  or  discounts  to  the 
customer.  This  cost  represents  the  true  cost  to 
process  a  requirement  and  may  or  may  not  be 
equal  to  the  cost  charged  to  the  customer. 

To  demonstrate  how  the  existence  of 
information  from  the  UCM  can  assist  contract 
specialists  and  contracting  officers,  a 
comparison  of  the  costs  to  issue  a  $200,000 
delivery  order  under  a  General  Services 
Administration  schedule  versus  a  contract 
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valued  at  $200,000  shows  the  GSA  order 
costs  $2959.59,  while  the  contract  costs  a 
minimum  of  $18,086.40.  Providing  the 
administrative  cost  to  process  an  action  can 
lend  support  to  ensure  decisions  are  made  in 
the  best  interests  of  the  Government. 

THE  CHALLENGE  TO  INCLUDE 
ALL  OPERATING  COSTS 

In  typical  government  operations,  identifying 
the  true  cost  of  a  service  is  almost 
impossible.  Unlike  the  private  sector,  most 
public  sector  services  are  “free”  to  the 
customer.  Even  government  internal  service, 
which  has  a  visible  cost,  usually  concentrates 
on  the  direct  labor  furnished  for  a  project. 
Not  only  was  there  a  dearth  of  information 
within  our  own  department  regarding 
operations  performance  and  costs,  but  also 
within  the  established  budget  and  finance 
organizations.  Many  of  the  costs  were  not 
known  because,  at  least  for  the  organization 
in  question,  the  costs  did  not  exist  as  an 
actual  cost  of  business. 

The  first  challenge  was  to  identify  all  labor 
costs  through  forecasts  of  workload  for  the 
coming  year.  Year-end  workload  reports  are 
reviewed  for  total  awards  in  each  workload 
category,  as  well  as  the  net  change  of  work- 
in-process  for  new  contract  awards.  The 
totals  are  adjusted  based  on  market  conditions 
to  determine  what  type  of  work  will  be 
coming  in  and  in  what  number.  An  example 
of  an  adjustment  we  made  is  in  contract 
administration.  With  NAWCWPNS  experi¬ 
encing  a  decrease  in  overall  funding,  many 
customers  find  they  must  downscope 
research  and  development  or  service 
contracts.  Delays  in  receipt  of  funds  or 
otherwise  stretching  efforts  cause  greater 
efforts  to  be  devoted  to  contract  admin¬ 
istration.  For  this  reason  adjustments  have 
been  made  to  increase  the  amount  of  time 
required  for  basic  contract  administration  and 
in  forecasting  the  number  of  modifications. 

A  model  was  created  to  calculate  the  amount 
of  labor  required  to  process  each  type  of 
service  provided  based  on  its  estimated 
number  and  amount  of  labor  required  for 


each  category  for  the  coming  year.  Included 
was  contract  administration,  contract  close¬ 
out,  and  issuance  of  delivery  orders  against 
indefinite  delivery — indefinite  quantity 
contracts.  After  forecasting  the  expected  level 
of  workload  for  each  category  of  complexity, 
the  data  are  input  in  the  “control  panel,”  and 
the  model  is  run  to  compare  the  number  of 
direct  personnel  required  to  perform  the 
workload  versus  the  number  of  personnel  on 
staff  available  to  provide  the  services.  The 
amount  of  labor  required  to  provide  the 
estimated  services  is  calculated  as  a  percent  of 
labor  utilization — the  percent  of  the  whole 
used  to  generate  revenue  for  the  organization. 

That  percentage  of  the  direct  personnel  not 
earning  revenue  for  the  organization  must  be 
passed  along  as  an  indirect  cost  to  all 
customers  providing  revenue.  Each  calcu¬ 
lation  of  direct  hours  is  then  divided  by  the 
labor  utilization  factor  to  determine  the 
number  of  hours  the  customer  must  fund  for 
the  department  to  meet  the  revenue  objective. 
The  number  of  hours  was  used  as  the  data 
field  to  be  manipulated,  rather  than  labor  and 
non-labor  costs  per  hour,  because  it  was 
important  for  the  rate  of  individual  costs  to  be 
visible.  Maintaining  the  same  costs  per  hour 
and  simply  identifying  additional  hours  as  net 
non-revenue-generating  hours  provided  the 
same  visibility  to  elements  of  costs,  as  well 
as  granting  visibility  to  the  under  utilization 
of  labor. 

Another  cost  recently  included  in  the  UCM 
was  the  cost  upgrade  and  replacement  of 
computer  hardware  and  software  resources. 
This  business  cost,  although  recognized, 
typically  has  not  been  considered  in  our 
department  budget  as  an  operations  cost.  In 
the  past,  funding  for  most  improvements  in 
computing  capability  were  provided  from 
headquarters  activities,  and  no  real  identifica¬ 
tion  of  the  year-to-year  replacement  costs  was 
made.  However,  the  department  recently 
identified  the  cost  of  maintaining  the  com¬ 
puting  capability.  This  cost  was  converted 
into  an  hourly  rate  based  on  the  total  number 
of  available  hours  by  direct  performing 
individuals  and  included  in  the  non-labor 
rate. 
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OTHER  FEATURES  OF 
DEVELOPMENT 

Ease  of  Use 

Ease  of  use  of  the  model  by  its  users  was  a 
necessary  feature  in  our  decentralized 
organization.  No  resources  were  available  to 
input  data  on  a  full-time  basis,  so  the  task  of 
providing  cost  estimates  to  customers  and  the 
Contracts  Department  had  to  be  integrated 
into  existing  functions.  Taken  as  a  whole,  the 
computing  skills  of  our  personnel  were  at  the 
most  basic  level.  Effective  implementation 
depended  on  enabling  persons  with 
procurement  knowledge  to  enter  the 
“procurement-based  knowledge”  in  a  manner 
requiring  little  computing  expertise.  At  the 
suggestion  of  High  Performance  Systems, 
Inc. — manufacturers  of  the  I  Think  software 
used  for  UCM  application — a  “control  panel” 
(Fig.  1)  for  data  entry  was  created.  The 
“buttons”  displayed  on  the  panel  are  the 
sources  of  data  input  for  use  in  the  I  Think 
model.  Entry  of  the  values  provided  the  entry 
for  each  occurrence  of  the  particular  cost 
entry,  no  matter  what  process  was  to  be 
evaluated.  The  control  panel  provides  an 
explanation  of  the  function  of  each  value, 
which  identifies  procurement  features 
indicating  additional  process  complexity. 

Ease  of  Variation  in  Model  Scenarios 

Another  important  feature  of  the  UCM  is  that 
inputs  can  be  changed  easily  to  show 
customers  differences  in  costs  based  on 
varying  levels  of  customer  service,  different 
types  of  direct  labor,  or  other  complexity 
factors  affecting  price.  The  model  enhances  a 
customer’s  understanding  of  the  factors 
affecting  cost  and  PALT  with  a  graphical 
depiction  of  inputs  and  outputs. 

REACTION  IN  THE  FINANCIAL 
COMMUNITY 

Some  in  the  financial  community  have 
maintained  that  use  of  a  “flat  tax”  would  be 
much  simpler  to  administer  than  this 
complicated  model  with  so  many  factors 
influencing  cost.  Evaluation  of  a  flat  tax, 
where  each  customer  is  charged  based  on  a 
flat  percentage  of  the  estimate  of  the  value  of 


the  procurement,  shows  that  the  customer 
with  the  small  procurement  is  vastly  under¬ 
charged  the  cost  of  executing  his  procurement 
action,  while  the  customer  with  large 
procurements  is  overcharged  and  indeed  ends 
up  subsidizing  the  small  dollar  procurements. 
Another  problem  with  a  flat  tax  is  that  quite 
often  excess  revenues  must  be  returned  at  the 
end  of  the  year,  because  the  tax  has  generated 
a  windfall  of  revenue  from  customers  with 
multimillion-dollar  requirements.  Programs 
suddenly  finding  themselves  with  extra  funds 
at  the  end  of  the  year  must  quickly  find 
vehicles  to  obligate  the  funding.  The  UCM 
approach  of  charging  the  customer  based  on 
the  actual  value  received  promotes  effective 
consolidation  of  requirements,  greater  use  of 
indefinite  delivery  indefinite  quantity  (IDIQ) 
contracts,  and  can  use  the  procurement  and 
technical  customer  labor  more  efficiently  than 
will  the  award  of  many  separate  small  dollar 
contracts. 

A  flat  tax  misrepresents  to  customers  the  true 
cost  to  process  requirements  and,  therefore, 
does  not  meet  this  key  objective  of  the  DBOF 
initiative.  There  is  little  chance  that  a  large 
program  office  will  want  to  find  another 
activity  to  procure  its  requirement  because  the 
administrative  cost  of  the  procurement  will  be 
appropriately  valued. 

These  arguments  convinced  NAWCWPNS 
management  that  we  should  proceed  with  the 
development  and  implementation  of  the  UCM 
versus  a  flat  tax  on  each  customer.  However, 
paradigms  within  the  Comptroller’s 
organization,  based  on  poor  experience  with 
service  centers,  have  to  date  made  it 
impossible  to  implement  a  fee-for-service 
program  based  on  the  costs  as  identified  by 
the  UCM.  The  Comptroller  has  been 
unwilling  to  risk  moving  from  G&A  funding 
to  the  fee-for-service  concept,  based  on  the 
difficulty  in  forecasting  workload  and  the 
lack  of  validation  for  the  model  assumptions. 
While  the  Contracts  Department  has  not  kept 
complete  records  of  output  or  accurately 
forecasted  workload  in  the  past,  we  feel  that 
implementation  will  focus  personnel  on  more 
effective  performance,  bring  in  additional 
revenue  for  the  Center,  and  provide  the 
necessary  incentive  and  focus  to  measure 
output  and  service  performance.  Inclusion  of 
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a  factor  to  compensate  for  time  allocated  by 
direct  personnel  to  activities  that  do  not 
generate  revenue  also  will  provide  somewhat 
of  a  buffer  to  changing  workload  conditions. 

For  now  the  Procurement  Department 
management  has  opted  to  implement  the 
UCM  internally,  believing  the  improvement 
in  measurement  and  accountability  of  our 
staff  is  a  sufficient  enough  benefit  to  support 


the  extra  work  imposed  on  the  employees. 
Through  internal  implementation  and 
improved  methods  of  measurement,  we 
intend  to  provide  the  Comptroller  with  the 
information  necessary  to  support  future 
implementation  of  a  fee-for-service  vehicle  to 
support  Procurement  Department  operations. 
The  two  graphs  show  a  typical  workload  and 
a  comparison  of  fee  for  service  versus  flat 
tax,  respectively. 


TYPICAL  WORKLOAD  DISTRIBUTION 
(CHINA  LAKE) 


SELLING  THE  CONCEPT 
INTERNALLY 

The  major  challenge  for  implementation  has 
been  in  communicating  the  need  for  change 
and  the  responsibility  of  each  person  in  the 
organization  to  do  so.  The  project  had  the 
support  of  top  management  from  the 
beginning.  For  the  line  managers,  it  was  a 
different  story.  These  managers  were  the 
most  familiar  with  the  problems  of  too  few 
resources  trying  to  manage  too  many  changes 
in  regulations  and  too  many  customer 
demands  and  deadline  pressures.  The  last 
complication  the  management  team  wanted  to 
deal  with  was  additional  reporting  require¬ 
ments  imposed  on  their  personnel.  Imple¬ 
mentation  from  top  management  down  was 
the  only  feasible  approach  for  a  relatively 
quick  implementation. 

Internal  briefings  and  demonstrations  of  the 
model  were  held  for  the  management  team 
and  small  groups  of  employees  to  introduce 
the  UCM  concept  and  explain  what 
implementation  would  mean  to  our  way  of 
operating.  At  the  briefings  the  employees 
were  able  to  air  their  fears  and  concerns,  such 
as  implied  performance  standards  and  risks 
associated  with  charging  customers  per 
service  rather  than  working  under  G&A 
funding. 

During  the  internal  briefings  employees  also 
were  trained  on  the  use  of  validation  forms 
(Fig.  2)  for  completion  and  submission  to  the 
project  office  following  execution  of  a 
procurement  service.  The  forms  were 
designed  to  collect  the  complexity  informa¬ 
tion  not  collected  in  any  of  the  manual  or 
automated  systems  until  implementation  of 
the  UCM.  The  validation  forms  were 
replaced  at  China  Lake  in  November  1994  by 
implementation  of  Milestone  Events  used  in 
the  Automated  Procurement  and  Accounting 
Data  Entry  (APADE)  system.  The  Milestone 
Events  are  used  by  contract  specialists  to 
indicate  status  of  a  procurement  requirement 
and  the  complexity  of  the  requirement. 
Implementation  of  APADE  milestones  has 
improved  our  internal  management  informa¬ 
tion  system  and  is  providing  critical 
information  for  management  of  workload. 


The  primary  impetus  for  development  of  the 
UCM — the  DBOF  initiative— has  not  yet  been 
implemented  universally  across  DOD. 
Implementation  at  NAWCWPNS  currently 
exists  at  only  a  top  level  within  the  Corporate 
Budget  office,  not  greatly  impacting  day-to- 
day  operations.  However,  if  we  are  asked  to 
fund  our  service  operations  through 
generation  of  operating  funds,  we  will  be 
ready.  In  the  meantime,  we  have  imple¬ 
mented  use  of  the  UCM  internally,  and  are 
beginning  to  gather  widespread  performance 
data  that  will  assist  in  the  validation  of  the 
assumptions  made  in  concept  development. 
We  also  are  using  the  cost  information  to 
communicate  with  our  customers,  so  they 
will  not  be  surprised  when  implementation 
occurs  outside  the  department.  Collection  and 
analysis  of  our  performance  and  operations 
data  are  beginning  to  meet  our  need  to  focus 
attention  on  the  way  we  process  our  work 
and  how  to  improve  our  processes. 

CHANGE  INFLUENCED  BY  THE 
UCM 

Implementation  of  the  UCM  has  influenced 
other  process  improvement/quality  efforts  in 
the  NAWCWPNS  Contracts  Department. 
Since  development  and  implementation,  the 
need  for  measurement  of  quality  and  output 
throughout  all  levels  of  the  organization  has 
had  a  greater  recognition.  An  improved 
understanding  has  been  gained  for  the  need 
for  a  comprehensive  staff  that  can  support 
such  a  project.  The  way  the  department  staff 
is  organized,  no  one  office  could  have 
developed  the  UCM.  Issues  with  the  UCM's 
use  and  data  elements  touch  on  areas  of 
administration,  policy,  or  automation.  As  a 
result  of  changes  both  within  and  outside 
DOD,  a  greater  realization  has  occurred  that 
our  staff  functions  can  no  longer  afford  to 
operate  in  separate  spheres  of  influence,  and 
changes  in  organizational  structure  are  being 
made  to  ensure  that  improved  teaming  will 
occur  on  future  projects.  Implementation  of 
the  UCM  has  provided  a  focus  on 
implementing  other  procurement  management 
efforts.  Information  identifying  the  com¬ 
plexity  of  work-in-process  must  be  available 
to  measure  the  amount  of  work  accomplished 
by  the  department.  Implementation  of  an 
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improved  information  system  was  accom¬ 
plished  in  part  because  of  the  need  for  the 
information  by  the  UCM. 

SUMMARY 

Development  and  internal  implementation  of 
the  UCM  has  provided  a  number  of  benefits 
to  the  Contracts  Department  at 
NAWCWPNS.  The  most  obvious  benefit  is 
that  we  are  prepared  for  implementation  of 
the  DBOF  initiative,  with  respect  to 
generation  of  operating  revenue,  with  a  well- 
thought-out  model  that  effectively  reflects  our 
costs.  The  effectiveness  of  implementation  of 
DBOF  or  some  other  fee-for-service  initiative 
will  not  be  degraded  by  a  hastily  created 


mechanism  for  generating  revenue.  Also,  the 
model  provides  some  much  needed 
performance  standards  for  the  department.  As 
we  continue  to  refine  our  cost  metrics,  we 
will  improve  our  management  of  scarce 
resources  required  to  provide  a  particular 
service.  Full  implementation  beyond  our 
department  will  provide  our  customers  with 
cost  and  schedule  information  to  effectively 
plan  and  budget  for  acquisition  requirements. 
Agreement  of  appropriate  levels  of  customer 
service  between  Contracts  Department 
personnel  and  technical  customers  will  spell 
out  the  expectations  for  performance  and 
revenue  for  both  parties.  Finally,  the  UCM 
provides  management  information  essential 
for  effective  department  operations. 
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CONTROL  PANEL  F 


For  Data  Entry,  Double  "Click”  on  each  Circle,  or 
"Button"  on  the  respective  panel,  and  enter  the 
appropriate  value  indicated  for  the  desired 
circumstance.  Following  entry  of  all  data, 
choose  the  corresponding  table  similar  to  that 
pictured  immediately  to  the  right,  Table  10, 
and  located  within  each  panel. 


Model 

Parameters: 
4X8  pages. 

4  Separate 
Modules,  each 
representing  a 
whole  system. 
Data  entry 
through  control 
panels  on  the 
left  side, pages 
1-5.  Model 
operations  at  rt. 


o 

Dollar  Value  Weighting 

Choices  for  $$  Value: 

D.O.  to  25K  : 

50 

D.O.  >25K  to  500K:100 

>50K  to  500K: 

100 

>500K  to  1M: 

150 

>1M  to  10M: 

200 

>10M  to  SOM: 

300 

>50M: 

400 

o 

Type  of  Contract 


Choices  for  Type: 

FFP  and  T&M:  0 
Cost:  1 00 

SB1R :  0 

(Total  Value:100) 
Not  a  Contract:  1 


O 

Required  Award  Date 


Less  than  Predicted  Lead  Time 
Available  1 

Acq  Pkg  Submitted  Within  Predicted 
Lead  Time,  or  Not  Applicable  0 


Basic  Hours 


Contract:  110 

Delivery  Ord.:  6 

SBIR:  40 

Streamlined  Pur:  6 


Total  Level  of  Customer  Service 


Choices  for  Desired  Level  of 
Service  (Premium  Effort  Over 
What  is  Required): 

Basic  Processing:  0.00 


Level  1  Service: 
Level  2  Service: 
Level  3  Service: 
Level  4  Service: 


Level  of  Customer  Service 
Required  Due  to  Requirement 
Complexity: 

Commercial  or  P/N  Item:  0.00 
FP  Non-Commercial 
Supply: 

0.02 

T&M/FP  Services:  0.05 

RDT&E: 

0.08 

Competitive  Serv  Requiring 
Cost  Realism  Analysis:  0.10 


Number  of  Contract  Awards 


I 


Number  of  Awards 

Initial:  1.0 

Subsequent:  0.7 


I 


Cancellation  Cost  Factor 


Up  to  Solicitation  Issuance  0.3 
Through  Pre  Bus  Clearance  0.6 
After  Negotiation  0.75 

Through  Post  Bus  Clearance  0.9 
Not  Applicable  1.0 


Requirement  Submission  Date  for  Same  Year  Award 


First  Qtr 


Second  and  Third  Qtr  2 


Fourth  Qtr 


Not  Applicable 


Fig.  1.  Control  Panel  Data  Entry. 


OR  CONTRACTS 


o 

Complexity  Factors 


Complexity  Choices  (Total  of 
All  Applicable  Elements): 

None: 

Other  Than  Full  and  Open 
Competition  or  8(a):  35 

Greatest  Value  Source  Sel:  100 

Competitive  Service  (Not  Greatest 
Value): 

50 

FIPR  >  $300K: 

Fixed  Price  RDT&E: 

Preproposal  Conf 
Classified  (Not  254): 

Consolidated  Requirements  or 
Performance  at  Multiple 
Govt  Sites  (Multiple  Users):  50 
ARP  Team  Assigned  50 


o 

Type  of  Solicitation 


Written 

1.0 

Oral 

0.7 

I 


Contract  Amount 


Enter  Specific 
Amount  For 
Determining 
Predicted  PALT 


o 

PALT  Adj  Factor 


Within  Quoted  Range  Plus  10%  1 

Outside  Quoted  Range  Plus  10%  .8 


o 

Ave  Cost  per  CS  Hour 


Requirement  Origination: 
NAWCWPNS :  $29.90 

Other  $38.96 


o 

Contract  Description 


R&D  46.4607 

Supplies  19.8525 

Service  0 


o 

Source  of  Rqmt  Origination 


NAWCWPNS  1 
Other  2 


o 

Type  of  Direct  Labor 


o 

Type  of  Delivery  Order 

FP  :15  FP  Requiring  Mkt  Survey:  200  Cost:  150  Not  a  D.O.:  0 


Instrument  ID 


Enter  Stub# 
for  ease  in 
tracking 


Note  Only: 

If  NAWCWPNS  Origination 
of  Requirment  then  rate  for 
Support  Staff  equals  24.65. 
Other  than  rate  equals 
$34.58  (Rate  for  Support 
Depts  as  publicized  by  the 
FY  95  Stabilized  Rates. 


Table  for  File  Copy  of  Contract 


Contract  PreAward 


Table  for  File  Copy  for  DO 


m 

Delivery  Order 


Table  of  Contract  Inputs 


Table  of  DO  Inputs 


VALIDATION  OF  CONTRACT  MODULE  OF  THE  UNIT  COST  MODEL 

An  effort  is  being  made  to  test  the  Unit  Cost  Model  that  predicts  the  cost  to  issue  a  contract,  purchase  order,  etc. 
If  you  provide  the  following  information,  you  will  assist  in  validating  the  model. 


SOLICITATION  NUMBER 


I  CONTRACT  SPECIALIST 


i  TECHNICAL  CODE 


ESTIMATED  DOLLAR  VALUE 


I  CONTRACT  TYPE 


REQUIRED  AWARD  DATE  DATE  ACCEPTED  BY  PROCUREMENT 


COMPLEXITY  ELEMENTS  FOR  THIS  REQUIREMENT  (Check  all  that  apply) 


ELEMENTS 


|  EXCLUSION  OF  SOURCES  {J  &  A  USED) 


!  PREPROPOSAL  CONFERENCE 


i GREATEST  VALUE  SOURCE  SELECTION 


!ORAL  SOLICITATION 


!  SPLIT  OR  BAA  AWARD 


[SERVICE  REQUIREMENT 


INITIAL  [ _ I  SUBSEQUENT 


CLASSIFIED  ATTACHMENTS  (Not  DD  254) 

;FIPR  REQUIRING  LIFE  CYCLE  COST  EVALUATION  OR  DESIRABLE  FEATURES 
|  PREAWARD  SURVEY  REQUESTED 
FIXED  PRICE  RDT&E  REQUIREMENT 
iREQUIREMENTS  FOR  MULTIPLE  SITES 


lARP  TEAM  ASSIGNED 


TO  BE  COMPLETED  BY 

CONTRACT  SPECIALIST  (Estimate  number  of  hours  required  for  processing) 
PROCESSING 


ACTUAL  DIRECT  CS  HOURS  TO  PROCESS  (Up  to  and  including  award) 


ACTUAL  CUSTOMER  SERVICE  HOURS  PROVIDED  FOR  THIS  REQUIREMENT  (Include  milestone 
meetings ,  other  meetings,  research  hours,  etc.)  (Purpose  of  effort  is  to  provide  additional  service  to 
customer.)  ABOVE  WHAT  IS  REQUIRED  TO  PROCESS  THE  REQUIREMENT. 

'LEVEL  OF  SERVICE  QUOTED  CUSTOMER 


ACTUAL  PROCUREMENT  ADMINISTRATIVE  LEAD  TIME  (DAYS) 

Problems  encountered  in  the  award  of  this  requirement  that  are  not  captured  by  the  above  complexity  elements: 


NAWCWPNS  4330/(T1)  (2-95) 


Return  your  completed  form  and  any  other  comments  to  Unit  Cost  Model  Program  Manager, 
2T0000D  (C6505).  Your  support  and  assistance  is  appreciated. 
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STUDY  TO  DETERMINE  THE  LIFE-CYCLE  COSTS  OF  PURCHASE  METHODS 

Laura  A.  Exley,  Procurement  Analyst,  Contracts  Department 
Naval  Air  Warfare  Center  Weapons  Division,  China  Lake,  California  93555-6001 


ABSTRACT 

The  China  Lake,  California,  site  of  the  Naval 
Air  Warfare  Center  Weapons  Division 
(NAWCWPNS)  is  chartered  to  support  the 
research,  development,  test,  and  evaluation 
(RDT&E)  requirements  of  Navy  weapon 
systems.  A  key  component  of  any  effective 
RDT&E  organization  is  the  ability  to  move 
rapidly  from  concept  to  development  to 
deployment.  Any  lead  time  that  can  be 
reduced  before  deployment  in  the  field 
improves  force  readiness,  and  any  savings 
accrued  in  administrative  costs  allow 
increased  funds  to  be  designated  for  technical 
support. 

In  an  effort  to  improve  procurement  support 
to  the  technical  customer,  the  management  at 
NAWCWPNS  China  Lake  supported  the 
development  and  implementation  of  two 
innovative  procurement  methods,  the  Small 
Purchase  Electronic  Data  Interchange 
(SPEDI)  and  Bankcard.  The  SPEDI  program 
is  based  on  requirements-type  contracts  using 
the  Just-in-Time  warehousing  and  delivery 
concept.  SPEDI  orders  are  placed  via 
personal  computer  and  feature  on-line 
catalogs  and  search  capability.  SPEDI  orders 
are  restricted  to  requirements  that  do  not 
exceed  the  small  purchase  threshold.  The 
Bankcard  program  uses  a  commercial  credit 
card  to  enable  customers  to  procure 
requirements  that  do  not  exceed  $2500. 

These  innovative  small  procurement 
methods,  in  addition  to  the  traditional 
Simplified  Purchase  and  supply  system  req¬ 
uisitioning  methods,  are  in  use  at  China 
Lake.  However,  as  resources  became  more 
limited,  a  study  to  assess  the  cost- 
effectiveness  of  each  method  became 
imperative.  The  objectives  of  the  study  were 
to  provide  management  with  a  better  under¬ 
standing  of  the  costs  associated  with  each 
method  to  promote  the  maximized  use  of  the 
least  costly  methods,  when  appropriate,  and 


to  enhance  effective  allocation  of  department 
resources. 

This  study  was  concerned  primarily  with  the 
effectiveness  of  each  procurement  method  as 
measured  only  by  the  life-cycle  cost  to 
process  requirements.  That  is,  the  cost  of  all 
procurement  processing  costs,  plus  the  cost 
associated  with  processing  the  requirement 
external  to  the  Contracts  Department.  A  life- 
cycle  concept  was  chosen  to  show  true 
representation  of  cost  rather  than  indicating 
costs  incurred  by  the  Contracts  Department 
alone.  This  concept  also  served  to  eliminate  a 
skewed  measurement  of  effectiveness.  The 
results  of  the  final  study,  which  indicated  the 
cost  to  process  each  requirement  line  item, 
are  as  follow: 


Bankcard  $26.39 

Simplified  purchase  $174.60 

SPEDI  $97.18 

Supply  system*  $270.38. 


*  Includes  the  cost  of  warehouse  operations 
and  procurement. 

Realizing  that  a  purchasing  method  with  an 
array  of  complex  procurement  issues  cannot 
be  addressed  by  cost-effectiveness  alone,  we 
addressed  the  goals  of  each  method  and  its 
target  markets.  At  the  core  of  the  study  was 
the  premise  that  each  purchasing  method  was 
a  valid  concept  developed  to  address 
particular  needs  of  the  customer  and  the 
procurement  community. 

This  paper  summarizes  the  methodology  used 
by  the  study  team,  the  study  results,  and  how 
the  study  is  being  used  at  NAWCWPNS. 

INTRODUCTION 

In  reviewing  the  effectiveness  of  processes, 
most  organizations  look  only  within  their 
internal  boundaries  for  definition  of  the 
process.  This  limited  view  of  the  scope  of  a 
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process  ignores  pertinent  costs  and  other 
measures  of  effectiveness.  The  procurement 
process  within  the  Federal  Government  is 
influenced  by  a  number  of  factors  external  to 
the  procuring  organization.  Factors  outside 
the  individual  procuring  organization  that  can 
and  will  affect  the  amount  of  lead  time 
required  to  procure  a  requirement  include 
adequacy  of  requirement  definition,  market 
forces,  and  changes  in  the  regulations  or  their 
enforcement. 

To  assess  how  the  Contracts  Department  at 
NAWCWPNS  should  allocate  resources  in 
support  of  systems  to  effect  acquisition  of 
needed  supplies,  we  decided  to  perform  a 
study  of  the  cost  of  each  method  used  by  the 
department  and  its  customers.  Thus,  the 
initial  study  to  determine  the  life-cycle  costs 
to  process  a  small  purchase,  SPEDI,  or 
Bankcard  order  was  chartered. 

The  advantages  of  performing  a  study 
unencumbered  by  organizational  boundaries 
are  that  the  entire  process  can  be  compared 
side-by-side  for  similarities  and  differences. 
A  resulting  difference  in  cost  can  prompt  a 
manager  to  question  the  differences  between 
methods  used  by  the  same  office  and  whether 
the  differences  are  appropriate. 

METHODOLOGY 

A  procurement  analyst  was  teamed  with  a 
management  analyst  to  interview  the  procure¬ 
ment  process  owners,  as  well  as  participants 
in  other  departments,  including  technical 
personnel  as  requirement  originators; 
Bankcard  holders;  cost  accounting/ 
comptroller;  and  plant  account  and  supply 
personnel. 

The  initial  assignment  seemed  rather  straight¬ 
forward,  but  the  team  soon  realized  that  a 
study  of  costs  in  a  process  often  brings  out 
feelings  and  emotions  rather  than  facts  and 
data.  The  two  innovative  procurement 
methods.  Bankcard  and  SPEDI,  both  having 
dedicated  personnel  believing  in  the 
"rightness"  of  their  programs,  were  often 
competing  with  each  other  for  customers  in 
the  same  market.  A  great  deal  of  emotion  had 


to  be  sorted  through  to  determine  the  best 
possible  estimate  of  costs,  given  all  the 
available  information. 

We  began  the  study  in  February  1992.  Our 
goal  was  to  determine  processing  costs  for 
Fiscal  Year  (FY)  1991  for  the  three 
procurement  methods.  Almost  immediately 
we  realized  that  FY  91  was  not  an  ideal  year 
on  which  to  base  future  decision-making. 
Each  method  had  unique  situations  that 
would  distort  costs.  Bankcard  was  in  its  first 
year  of  transition  from  a  manual  process  to  an 
automated  process.  Even  though  particular 
attention  was  given  to  the  needs  of  the 
Accounting  Branch,  a  significant  number  of 
rejects  caused  backlogs  of  rework  for 
accounting  personnel.  This  rework  caused 
accounting  costs  to  soar  while  accounting 
technicians  and  technical  Bankcard  users 
were  learning  the  process.  SPEDI  was  still  in 
early  development,  having  had  its  initial 
contract  awarded  and  in  place  for  only  9 
months.  Contracts  personnel  in  the  SPEDI 
Division  were  still  developing  internal 
processes  and  methodologies  for  solicitation 
and  award  for  various  types  of  commodities. 
A  significant  portion  of  the  man-years 
dedicated  to  this  program  were  used  to 
perform  this  research  and  development  effort. 
The  small  purchase  method,  along  with  the 
newly  automated  Bankcard  method,  had  to 
bear  all  the  costs  for  operation  and 
maintenance  of  the  Tandem  computer,  even 
though  major  contracts  would  soon  be 
automated  and  would  share  in  allocation  of 
this  cost  element. 

For  each  method,  it  was  natural  to  believe 
that  FY  91  represented  a  transitional  year  and 
that  any  study  proposing  to  serve  the 
information  needs  of  the  department  would 
need  to  forecast  beyond  that  time.  From  an 
early  briefing,  management  approval  was 
obtained  to  conduct  the  study  of  FY  91  and 
the  forecast  for  FY  92  and  FY  93. 

Where  possible,  process-specific  costs  were 
identified.  Often  personnel  and  resources  are 
assigned  to  a  variety  of  processes,  so  their 
costs  are  shared  on  a  pro-rata  basis. 
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GATHERING  THE  DATA 

A  sample  of  Bankcard  holders  and  approving 
officials  with  varying  levels  of  experience 
were  interviewed  to  determine  the  average 
time  spent  processing  credit  card  purchases, 
reconciling  statements  at  the  end  of  each 
month,  and  approving  purchases.  Small 
purchase  buyers  spent  100%  of  their 
available  time  processing  small  purchases, 
unless  other  work  assignments  significantly 
affected  their  availability  to  process  small 
purchase  requirements.  For  example,  some 
of  the  purchasing  agents  also  served  as 
Bankcard  representatives  to  their  technical 
customers. 

Approximately  30%  of  the  average  Bankcard 
representative's  time  was  spent  assisting 
customers,  fielding  questions  on  the 
application  of  regulations,  and  performing 
monthly  audits  of  purchases.  Thus,  this  time 
would  be  considered  a  cost  to  the  Bankcard 
method  rather  than  to  the  small  purchase 
method. 

In  the  initial  interview,  participants  often  did 
not  know  how  much  time  was  required  for 
processing.  Persistence  in  this  line  of 
questioning  (sometimes  offering  suggestions, 
comparisons,  and  finally  additional  ques¬ 
tioning)  obtained  an  estimate  of  the 
participant's  "gut  feel."  Without  this  follow¬ 
up  from  the  team,  some  individuals  would 
have  just  given  up  because  they  were  not  able 
to  quantify  their  costs  on  their  own.  One 
probable  explanation  for  this  situation  is  that 
personnel  in  the  Contracts  Department  are 
trained  to  work  with  great  detail  and  precision 
regarding  contract  and  Simplified  Purchase 
costs.  Such  training  does  not  easily  lend  itself 
to  making  an  estimate  of  cost  to  be  used  for 
decision-making,  especially  when  no 
management  information  is  in  place  to  capture 
supporting  data. 

Management  personnel  were  queried  to 
assess  the  amount  of  time  spent  in  support  of 
each  method.  This  time  varied  from  10  to 
100%,  so  each  individual's  estimated  time 
was  addressed  separately.  Actual  labor  costs 
were  used  when  available.  Otherwise, 
projected  labor  expenses  were  used. 


Contractor  support  services,  primarily  in  the 
areas  of  data  processing  and  computer 
operations,  were  estimated  to  be  continued  at 
the  same  level  for  all  years,  except  where 
changes  in  processes  had  eliminated  the  need 
for  some  types  of  support.  The  costs  for 
these  services  were  obtained  from  the 
technical  monitor  for  the  appropriate  delivery 
order  providing  the  service. 

After  assimilating  all  the  data,  the  team  often 
had  to  evaluate  the  individual  responses  and 
the  given  circumstances  of  each.  In  case  of 
conflicting  data,  some  responses  were  given 
more  weighting  than  others  based  on  the 
probability  of  that  occurrence  representing  the 
normal  experience. 

EXAMINATION  OF  THE  DATA 

The  team  had  multiple  meetings  and/or 
telephone  contacts  with  personnel  involved  in 
using  each  method  to  obtain  clarification  or 
additional  data  regarding  processes  and  costs. 
These  contacts  also  served  as  a  check  and 
balance  for  the  study,  as  the  representatives 
tended  to  scrutinize  time  and  other  cost 
assessments  for  the  other  methods  as  well  as 
their  own.  The  team  deliberately  used  the 
dynamics  created  by  the  examination  of  costs 
to  break  through  early  assumptions  and  to  get 
closer  to  the  representative  costs. 

Examination  of  the  data  by  the  interested 
parties  also  served  to  educate  them  to  the 
costs  allocated  to  specific  areas  of  the 
process.  Sometimes,  the  information 
provided  was  the  exact  opposite  of  the 
paradigm  a  person  held.  For  instance,  it  was 
believed  that  Bankcard  automation  would 
improve  the  cost-effectiveness  of  the  plant 
account  process.  In  fact,  Bankcard  auto¬ 
mation  dramatically  improved  the  efficiency 
with  which  plant  account  items  were 
identified.  However,  due  to  improved  iden¬ 
tification  measures,  the  number  of 
acquisitions  requiring  plant  account  screening 
had  increased  enough  to  require  additional 
labor  to  perform  the  function.  Savings  were 
expected  at  the  next  major  triennial  inventory 
due  to  fewer  losses  of  plant  property,  but  the 
exact  savings  could  not  be  quantified  at  the 
time  of  the  study. 
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FORECASTING  CUSTOMER 
DEMAND 

When  we  began  our  initial  study  of 
Simplified  Purchase  methods,  FY  91 
represented  the  only  year  to  be  studied  that 
had  complete  data.  To  calculate  costs  for  FY 
92  and  FY  93,  we  had  to  forecast  the  likely 
incoming  workload  for  each  method.  Using 
the  same  level  of  workload  for  each  year 
would  have  been  easier,  but  we  believed  it 
appropriate  to  adjust  the  level  of  workload 
based  on  trends  we  had  seen  taking  place. 
Traditional  Simplified  Purchase  workload 
was  steadily  eroding,  while  Bankcard  was 
being  used  more  frequently  each  year.  The 
SPEDI  Division  was  actively  reviewing  both 
Simplified  Purchase  and  Bankcard  reports  for 
demand  data,  indicating  likely  candidates  for 
inclusion  in  an  upcoming  SPEDI  contract. 
Because  the  customer  base  for  each  method 
was  in  a  dynamic  state  of  change,  showing 
the  effects  of  expected  trends  on  the  life-cycle 
cost  to  process  requirements  by  forecasting 
the  customer  demand  seemed  the  most 
appropriate  way  to  arrive  at  realistic  costs. 

Forecasting  proved  to  be  one  of  the  most 
challenging  demands  of  the  study.  The  major 
factors  considered  were  as  follows: 

1.  Specific  SPEDI  contracts  anticipated 
to  be  available  for  use  by  the  customer, 
anticipated  customer  volume,  and  percent  of 
the  fiscal  year  available  for  use  after  contract 
award  and  start-up  lead  time. 

2.  Effect  of  decreasing  Department  of 
Defense  and  NAWCWPNS  budgets  on 
demand  for  all  methods. 

3.  Effect  of  increased  numbers  of 
SPEDI  requirements- type  (mandatory  use) 
contracts  on  demand  for  small  purchase  and 
Bankcard  business  bases. 

4.  Number  of  personnel  expected  to  be 
employed  at  NAWCWPNS,  which  would 
have  a  significant  impact  on  the  office  supply 
contract  for  SPEDI,  the  source  of  a 
significant  number  of  orders. 


LIFE-CYCLE  COSTS  OF 
PROCESSING  REQUIREMENTS 

The  following  table  provides  our  estimate  for 
FY  91  life-cycle  costs  to  process  a  require¬ 
ment  by  each  method  on  a  per  line  item  and 
per  order  basis.  The  costs  indicated  are  not  a 
judgment  indicating  what  each  method  should 
cost,  but  merely  our  best  assessment  of  the 
actual  average  cost  to  process  a  requirement 
by  each  method,  as  experienced  by  the 
Contracts  Department  and  other  departments 
at  NAWCWPNS  China  Lake. 


Method 

No.  of 
line  items 

L/I 

cost,  $ 

No.  of 
orders 

Order 
cost,  $ 

Bankcard* 

51,841 

30.98 

23,564 

68.15 

Simplified 

purchase 

25,134 

178.35 

9,309 

481.57 

SPEDI 

11,243 

86.45 

11,243 

86.45 

*The  cost  of  the  Bankcard  method  is  based 
on  manual  processing  of  documentation  and 
end-of-month  reconciliation  of  the  bank  state¬ 
ment  of  purchases.  The  automated  method  of 
processing  documentation,  with  the  benefit  of 
automated  funds  approval  before  execution  of 
the  transaction,  became  operational  during 
FY  92. 

Fixed  and  Variable  Costs 

Fixed  and  variable  costs  provide  information 
regarding  which  processes  can  be  expected  to 
see  a  reduction  in  costs  due  to  increases  in 
volume.  Because  the  fixed  costs  associated 
with  a  process  tend  to  remain  stable  with  each 
increase  in  individual  orders  or  line  items 
processed,  the  fixed  cost  per  unit  decline. 
Variable  costs  tend  to  vary  with  each  increase 
or  decrease  in  units.  Thus,  on  a  unit  price 
basis,  the  fixed  costs  will  vary  with  the 
number  of  units  while  the  variable  costs  will 
remain  constant  no  matter  what  the  order 
volume  may  be. 

In  a  perfect  world,  labor  could  be  shifted 
easily  to  other  tasks  when  not  being  used.  In 
practice  we  found  that  most  of  our  costs  are 
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probably  semi-variable  and  semi-fixed.  In  the 
operation  of  government  activities,  personnel 
represent  a  fixed  cost.  Since  NAWCWPNS, 
in  effect,  will  experience  this  labor  cost  no 
matter  what  the  level  of  work  in  process  in 
Simplified  Purchase,  at  least  in  the  short 
term,  we  assigned  Simplified  Purchase  labor 
to  the  fixed  cost.  Why  an  argument  could  be 
made  to  consider  these  costs  as  variable  can 
easily  be  understood. 

Other  fixed  costs  for  Simplified  Purchase  are 
computer  support  for  operation  of  automated 
systems  for  accounting  and  procurement, 
analyst  support  of  the  Simplified  Purchase 
function,  Simplified  Purchase  management 
support,  operation  and  maintenance  of 
automated  systems,  and  plant  account. 
Variable  costs  include  the  cost  for  the 
customer  to  prepare  the  documentation  for  the 
requisition,  obtain  necessary  approvals, 
receiving  and  delivery,  and  accounting. 

Bankcard  and  SPEDI  costs  were  addressed  in 
a  similar  manner.  Both  these  methods  include 
operating  costs  unique  to  their  method.  A 
variable  cost  for  Bankcard  was  the  bank 
surcharge  fee,  the  rate  of  which  varied  at  the 
individual  order  level  at  any  given  time  based 
on  the  total  number  of  orders  placed  at  the 
time  processed  by  the  bank.  No  other  process 
experienced  a  bank  surcharge  fee  as  a  cost  of 
the  process.  Computer  access  charges  are 
included  in  the  SPEDI  variable  costs  as  they 
can  be  attributed  to  each  individual  order.  No 
other  method  incurred  such  access  charges 
from  NAWCWPNS  operations. 

The  following  table  provides  a  breakout  of 
fixed  and  variable  costs  per  line  item  for  each 
method. 


Method 

Fixed 
cost,  $ 

%of 

cost 

V  ariable 
cost,  $ 

%of 

cost 

Bankcard* 

6.03 

20 

24.95 

80 

Simplified 

100.51 

56 

77.84 

44 

purchase 

SPEDI 

60.19 

70 

26.26 

30 

Costs  per  line  item  and  per  order  for  fiscal 
years  1992  and  1993  were  estimated  to  be  as 
follows: 


Method 

1992 

1993 

Line 
item,  $ 

Order, 

$ 

Line 
item,  $ 

Order, 

$ 

Bankcard 

33.63 

73.99 

36.40 

80.07 

(Automated) 

Simplified 

179.54 

484.76 

163.70 

441.98 

purchase 

SPEDI 

95.24 

95.24 

90.46 

90.46 

NOTE:  The  ratio  of  average  line  items  per 
order  is  as  follows:  SPEDI,  1:1;  Bankcard, 
2.2:1;  and  Simplified  Purchase,  2.7:1.  The 
ratios  were  calculated  for  FY  91. 
Historically,  these  ratios  have  remained 
remarkably  consistent  over  the  last  several 
years,  and  were  used  in  our  calculations 
whenever  actual  data  were  not  available. 

MEASURES  OF  EFFECTIVENESS 
OTHER  THAN  COST 

The  cost  of  a  process  reflects  only  a  portion 
of  its  effectiveness  in  federal  acquisitions. 
Other  factors  for  consideration  are  the  ability 
of  the  method  to  meet  customer  needs;  the 
level  of  risk  to  procurement  authority  at  the 
activity  imposed  by  the  process;  and  the 
ability  of  the  process  to  meet  performance 
goals  established  at  inception  of  the  process. 
To  effectively  manage  resources  and  meet  the 
needs  of  customers,  managers  must  be  aware 
of  these  other  considerations.  Some  require¬ 
ments  may  need  to  trade  cost  efficiency  for 
other  considerations.  However,  we  found 
that  the  majority  of  requirements  processed  at 
NAWCWPNS  would  meet  the  characteristics 
of  the  target  markets  for  processing  by  cost- 
effective  methods  other  than  traditional 
Simplified  Purchase. 

Target  Markets 

In  performing  our  life-cycle  cost  study,  we 
never  intended  to  try  to  eliminate  any  Sim¬ 
plified  Purchase  method  at  NAWCWPNS. 
We  believed  that  each  method  served  valid 
customer  needs,  and  identification  of  the 
target  market  for  each  method  verified  this 
belief. 
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The  SPEDI  method  was  determined  most 
appropriate  to  procurement  of  high-volume, 
commercial,  off-the-shelf  products.  Because 
the  customer  has  no  direct  interaction  with  the 
vendor,  the  products  available  through  the 
catalog  must  meet  the  customer  requirement 
without  the  need  for  negotiation.  Inclusion  of 
high-demand  requirements  under  SPEDI 
contracts  promotes  additional  cost  savings 
through  negotiation  of  quantity  discounts. 
The  high  percentage  of  fixed  costs  associated 
with  process  automation  provides  SPEDI 
with  the  potential  for  very  low  average-per- 
order  costs  if  a  high  volume  of  orders  is 
placed  against  its  contracts. 

The  traditional  Simplified  Purchase  method  is 
most  appropriate  for  acquisition  of  those 
requirements  that  may  require  negotiation  of 
terms,  conditions,  or  specifications  for  items 
not  readily  available  for  off-the-shelf 
purchase.  The  flexibility  of  this  method, 
combined  with  a  high  dollar  threshold 
compared  to  the  Bankcard,  enabled  it  to 
provide  the  highest  return  on  operating  costs 
of  any  of  the  three  methods. 

The  Bankcard  is  effective  for  specialized, 
low-volume  items  that  do  not  exceed  the 
$2500  per  purchase  limitation.  Bankcard  also 
is  effective  for  commercial,  off-the-shelf 
items,  but  the  method  probably  has  already 
reached  its  optimum  cost  per  order. 
Increasingly,  the  Contracts  Department 
should  include  items  with  considerable 
customer  demand  in  appropriate  SPEDI 
contracts  to  reduce  the  average  prices  paid. 
The  low  per-line-item  cost  makes  this  method 
especially  attractive  over  other  methods  for 
requirements  within  its  low  dollar  threshold 
limitation. 

Risk  Assessments  of  Each  Method 

A  further  study  objective  was  to  assess  the 
procurement  risk  associated  with  the 
improper  use  of  each  method  studied. 
Procurement  risk  is  defined  as  the  probability 
that  fraud,  waste,  abuse,  or  noncompliance 
of  acquisition  regulations  or  policies  will 
occur  at  any  given  time  within  the  processes 
of  each  method.  We  found  that  the  SPEDI 
method  presented  NAWCWPNS  with  the 
least  amount  of  risk.  The  low  risk  inherent 


with  this  method  is  due  to  negotiation  of 
prices  with  volume  discounts  built  into  the 
basic  contract,  the  on-line  verification  of 
availability  of  funding  before  placement  of 
the  order,  and  concentration  of  competitively 
awarded  contracts  for  commodities  in  areas 
satisfied  by  commercial  items.  Furthermore, 
consolidation  of  customer  needs  into  a 
requirements-type  contract  provides  the 
Government  with  greater  remedies  than 
multiple  purchase  orders. 

The  built-in  system  of  manual  reviews  and 
checks  and  balances  present  in  processing 
Simplified  Purchases  provided  a  greater 
opportunity  for  human  error  than  did  the 
process  for  ordering  via  SPEDI.  The  number 
of  reviews  present  in  all  areas  of  the  process 
have  a  detrimental  impact  to  administrative 
lead  time,  causing  this  method  to  experience 
the  longest  administrative  lead  time  by  far. 
Though  this  method  has  greater  control  by 
procurement  personnel,  the  length  of  the 
processing  cycle  adversely  impacts  the 
customer  by  adding  schedule  and  market 
risks. 

The  Bankcard  method,  like  SPEDI, 
encourages  quick  processing  of  the 
requirement,  delivery,  and  payment  to  the 
vendor.  Automation  of  the  process  enables 
verification  of  funds  to  be  made  on-line  while 
processing  the  order,  saving  time,  and 
reducing  the  risk  for  each  requirement.  The 
primary  risk  associated  with  the  Bankcard  is 
that  the  cards  are  primarily  in  the  control  of 
and  use  by  technical  customers.  Because 
these  customers  are  not  full-time  procurement 
officials,  delegation  of  procurement  authority 
to  these  persons  outside  the  Contracts 
Department  carries  a  high  level  of  risk  of 
abuse  of  procurement  laws  and  regulations. 
The  relatively  high  level  of  risk  associated 
with  the  Bankcard  method  is  somewhat 
tempered  by  its  limitation  to  purchases  that  do 
not  exceed  $2500,  and  again  by  total  dollar 
limitations  on  a  total  monthly  basis. 

Performance  Goals 

Each  method  had  a  different  set  of  perform¬ 
ance  goals  its  processes  were  designed  to 
meet.  SPEDI  was  designed  to  greatly 
improve  the  response  time  from  initiation  of 
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the  acquisition  requirement  through  the  time 
of  receipt  of  the  item(s).  Other  SPEDI  goals 
were  the  reduction  of  total  procurement  effort 
by  consolidating  recurring  Simplified 
Purchase  requirements  and,  thereby, 
reducing  the  procurement  cost  for  small 
dollar  purchases;  achieving  lower  average 
pricing  on  requirements  due  to  negotiation  of 
volume  discounts;  improving  quality  of 
merchandise;  providing  easy  accessibility  to 
all  personnel;  lowering  levels  of  government 
warehouse  inventory;  reducing  inventory 
cost;  and  improving  vendor  payment  times. 

Goals  for  the  Bankcard  program  were  to 
improve  response  time  from  initiation  of 
acquisition  through  receipt  of  the  order;  serve 
as  a  cost-effective  alternative  to  the  traditional 
Simplified  Purchase  method;  empower 
technical  users  to  obtain  supplies  directly  for 
better  mission  support;  and  streamline 
payment  to  vendors. 

The  goal  for  the  traditional  Simplified 
Purchase  method  was  to  support  the  technical 
customer  through  the  procurement  of  all 
requirements' within  the  small  purchase  dollar 
limitation  at  fair  and  reasonable  prices  and  in 
accordance  with  all  applicable  laws  and 
regulations  in  effect. 

Identification  of  the  goals  of  each  method  is 
important  to  keep  the  role  of  each  method  in 
proper  perspective,  and  the  importance  of 
objectives  other  than  cost.  All  three  purchase 
methods  were  determined  effective  in  meeting 
their  respective  performance  goals. 

PRODUCT 

The  product  of  our  study  was  a  document 
that  will  serve  managers  of  Simplified 
Purchase  methods  at  NAWCWPNS  for  many 
years.  This  thorough  documentation  provides 
a  basis  from  which  future  follow-up  studies 
can  be  made  easily.  Critical  to  any  life-cycle 
cost  study  are  complete  documentation  of 
how  costs  were  determined,  the  base  of  allo¬ 
cation,  any  unique  circumstances  influencing 
the  costs,  and  the  source  of  the  information. 
From  such  documentation,  discussions  of  the 
relevancy  of  cost  estimates  can  be  held.  After 
a  determination  that  most  of  the  value-added 
effort  in  the  proper  identification  of  costs  was 


achieved,  the  documentation  properly  serves 
as  an  historical  record  for  future  studies  or 
comparisons. 

A  suggestion  that  discussions  and  reviews  of 
a  study  might  conclude  after  most  value- 
added  effort  have  been  accomplished  might 
seem  odd  to  some.  However,  from 
experience  I  have  found  that  in  a  search  for 
the  exact  cost  of  a  process,  you  might  never 
finish  the  study!  At  some  point  a  deter¬ 
mination  must  be  made  that  any  remaining 
refinements  of  cost  will  be  relatively  minor 
and  not  affect  the  relative  cost  standings  in 
question.  A  decision  to  conclude  the  study 
has  to  be  reached. 

PHASE  II— THE  SUPPLY 
DEPARTMENT  STUDY 

From  the  beginning  of  implementation  of  the 
SPEDI  program,  the  Supply  and  Contracts 
Departments  competed  in  the  pursuit  of  a 
number  of  common  items  for  processing 
through  SPEDI  and  the  Department  of 
Defense  Supply  System.  Disagreement  about 
what  items  must  legally  be  obtained  from 
inventory  through  the  supply  system  has  had 
two  major  impacts  to  the  SPEDI  program. 
First,  the  SPEDI  contracts  are  written  as 
requirements  contracts.  Therefore,  any 
requirement  covered  by  a  SPEDI  contract  that 
ends  up  also  being  procured  through  the 
supply  system  violates  terms  of  the  SPEDI 
contract,  and  ends  up  subjecting  the 
command  to  claims  from  the  injured 
contractor.  Second,  a  restrictive  interpretation 
of  regulations  that  unduly  limits  the  number 
of  items  available  for  purchase  under  SPEDI 
significantly  impedes  growth  of  the  program. 

With  a  relatively  high  percentage  of  fixed 
versus  variable  costs,  SPEDI  has  the  most 
potential  of  any  of  the  methods  to 
dramatically  reduce  its  per  order  costs  with  an 
increase  in  order  volume.  This  information  is 
critical  for  management  and  customers  alike, 
especially  considering  that  the  SPEDI 
contract  providing  the  highest  order  volume 
is  the  contract  for  office  supplies,  repre¬ 
senting  an  average  of  73%  of  the  total 
number  of  orders  against  vendor  contracts  in 
the  last  3  years.  As  would  be  expected  for  a 
contract  for  office  supplies,  the  average  dollar 
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value  for  an  individual  order  in  those  years 
was  $91.86,  making  it  vitally  important  to 
reduce  the  per  order  costs  for  these  low  dollar 
requirements. 

Central  to  the  argument  of  the  Supply 
Department  officials  was  the  belief  that  their 
process  was  more  efficient  and  less  costly  to 
the  customer.  After  the  initial  life-cycle  cost 
study  was  completed  in  January  1993,  the 
Supply  Department  requested  that  a  team 
work  to  study  supply  operations  so  that  an 
understanding  of  the  costs  associated  with  the 
processing  of  orders  would  be  available  to  all 
procurement  officials.  In  June  1993,  at  the 
request  of  NAWCWPNS  management,  a 
team  began  the  study  of  the  supply  processes 
and  associated  costs. 

Counting  Procurement  Line  Items 
Versus  Issues 

The  single  most  critical  issue  the  team  faced 
was  selecting  the  unit  of  measurement  to  be 
used  as  the  basis  for  distribution  of  costs. 
Supply  Department  personnel  proposed  to 
use  issues  from  their  warehouse  as  the  cost 
measurement  unit.  The  Contracts  Department 
representatives  believed  use  of  a  line  item  as 
the  basic  unit  of  measure  was  the  only  valid 
approach  because  this  unit  was  the  only  one 
common  to  all  methods  studied.  A  survey  by 
the  Supply  Department  determined  that  for 
FY  93  there  was  a  ratio  of  6.74  issues  per 
procurement  line  item  of  material  ordered.  By 
the  time  the  cost  study  team  was  chartered  to 
analyze  the  real  cost  for  processing  supply 
requirements,  the  debate  between  the  cost 
effectiveness  of  SPEDI  versus  issues  from 
the  supply  warehouse  had  been  ongoing  for 
about  3  years.  Both  groups  were  so 
convinced  of  their  own  cost-effectiveness  that 
they  failed  to  see  they  were  not  using 
equivalent  units  of  measure.  This  discovery, 
and  getting  our  supply  officials  to 
acknowledge  the  difference,  was  a  significant 
accomplishment  of  the  study.  The  bases  of 
allocation  did  not  impact  the  total  cost,  merely 
the  perception  of  how  expensive  a  process 
was.  The  issue  for  the  Supply  Department 
was  whether  costs  would  be  spread  over 


18,777  line  items  or  across  90,384  issues. 
For  the  average  person,  $40.12  per  issue 
sounds  much  more  cost-effective  than 
$270.38  per  line  item,  although  both  figures 
represent  the  same  actual  cost. 

Allocation  of  Elements  of  Cost 
Among  Methods 

Allocation  of  costs  for  the  various  methods 
was  based  on  the  percentage  of  the  total 
number  of  line  items  processed  using  that 
method  in  the  particular  cost  area.  Some  costs 
were  determined  to  benefit  more  than  one 
method  equally,  so  these  costs  were  shared 
equally.  The  primary  example  is  in  the  area  of 
computer  operations  support.  Following  the 
equal  allocation  per  method,  the  amount  was 
divided  by  the  total  number  of  line  items 
awarded  under  each  method. 

COST  OF  SUPPLY  AND  OTHER 
PROCUREMENT  METHODS 

After  20  months  of  discussing  costs  for 
supply  issues  and  system  requisition  orders, 
team  members  reached  no  agreement.  Aside 
from  whether  costs  should  be  identified  per 
issue  or  line  item,  the  primary  remaining  area 
of  disagreement  lay  in  the  area  of  contractor 
operations  in  support  of  the  Supply  Depart¬ 
ment  functions.  Support  for  calculation  of 
costs  for  elements  in  this  area  was  never 
provided  by  other  team  members,  so  this 
study  was  never  able  to  identify  exactly  the 
source  of  differences.  Virtually  all  the  supply 
data  had  to  be  gathered  manually  for  this 
study.  For  most  of  the  duration  of  the  study 
effort,  how  many  issues  had  been  made,  how 
many  line  items  of  requisitions  had  been 
processed,  etc.,  was  never  really  clear.  This 
lack  of  available  information  prolonged  the 
study  beyond  what  we  had  anticipated. 

Based  on  the  exhaustive  reviews  and 
discussions  held,  the  following  costs  for  FY 
93  represent  what  we  believe  to  be  the  life- 
cycle  processing  costs  for  each  method.  The 
costs  for  some  methods  have  been  revised 
based  on  new  information  discovered  during 
the  second  study. 


364 


Method 

No.  of 
line  items 

L/I 

cost,  $ 

No.  of 
orders 

Order 
cost,  $ 

Bankcard 

61,978 

26.39 

26,536 

61.65 

Simplified 

19,578 

174.60 

7,543 

453.17 

purchase 

SPEDI 

17,086 

97.18 

17,086 

97.18 

Supply* 

18,777 

270.38* 

N/A 

N/A 

operations 

Supply 

90,384* 

40.12* 

issues 

*No  figure  was  available  for  Supply 
Department  orders.  The  number  of  issues 
represents  the  total  number  made  at  the 
warehouse.  Cost  per  line  item  includes  cost 
for  MILSTRIP  orders  for  individual  use  and 
for  stock,  and  warehouse  costs  of  locally 
stocked  items. 


Method 

Fixed 
cost,  $ 

Fixed 

%of 

cost 

Variable 
cost,  $ 

Variable 
%  of 
cost 

Bankcard 

2.71 

10 

23.68 

90 

Simplified 

purchase 

97.78 

56 

76.82 

44 

SPEDI 

68.03 

70 

29.15 

30 

Supply 

51.44 

19 

218.94 

81 

Bankcard  has  achieved  a  low  fixed-cost  per 
line  item  based  on  its  large  volume  of  orders. 
Because  such  a  small  percentage  of  the  total 
cost  is  associated  with  fixed  costs,  the 
Bankcard  process  will  find  it  difficult  to 
greatly  reduce  its  cost  when  the  total  cost  is 
already  very  low.  The  automation  associated 
with  Simplified  Purchase  and  SPEDI,  as  well 
as  a  relatively  fixed  workforce,  provides  the 
fixed  costs  for  these  methods.  SPEDI  is 
particularly  well  positioned  to  lower  its  costs 
due  to  a  high  percentage  of  fixed  costs.  The 


high  variable  costs  associated  with  the  supply 
processes  reflect  an  absence  of  effective 
automation.  Though  most  of  the  processes  in 
FY  93  were  manual,  the  existing  automation 
was  determined  ineffective  and  required  con¬ 
siderable  manual  intervention  for  processing 
of  requirements.  Implementation  of  auto¬ 
mation  after  FY  93  was  expected  to  improve 
processing  efficiency  and  costs  after  the 
initial  start-up  period.  Future  studies  will  be 
made  to  capture  the  expected  cost  reductions. 

CONCLUSION 

The  benefits  of  performing  a  life-cycle  cost 
analysis  lie  in  understanding  the  nature  of 
costs  of  an  operation,  identifying  an  organi¬ 
zation's  leverage  points,  and  developing  an 
improved  capability  for  strategic  planning 
based  on  cost  efficiencies  and  market 
advantages.  The  perspective  we  obtained 
from  our  studies  was  that  an  exact  number 
can  be  elusive  due  to  a  number  of  estimates 
and  assumptions  that  must  be  made.  More 
important  was  the  understanding  that  the 
relative  standing  of  the  various  procurement 
methods  remained  constant,  no  matter  how 
many  times  a  particular  cost  element  was 
revised.  Allocation  of  costs  over  a  base  with 
many  thousands  of  units  requires  significant 
cost  changes  to  greatly  affect  the  unit  cost. 

You  may  feel  that  quantifying  some  costs  is 
just  not  possible;  therefore,  a  life-cycle  cost 
analysis  is  not  possible.  Every  organization 
probably  has  some  of  these  costs  not 
previously  recognized.  Through  the  use  of 
documented  logical  assumptions,  previously 
unquantified  costs  can  be  attributed  to  a 
service  or  a  product.  You  may  not  feel 
comfortable  using  assumptions  to  develop 
costs  that  will  form  the  basis  of  decision¬ 
making,  but  the  organization  will  be  better 
served  through  the  attribution  of  these  costs 
rather  than  simply  ignoring  them. 
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A  METHODOLOGY  FOR  ESTIMATING  BEST  VALUE 


Janine  L.  Faris 
Joan  S.  Lovelace 
The  MITRE  Corporation 


ABSTRACT 

The  methodology  described  in  this  paper 
takes  only  a  few  steps  beyond  the  Federal 
Information  Processing  (FIPS)  guidelines. 
The  term  “Best  Value”,  or  “Greatest  Value” 
has  no  statutory  or  regulatory  definition.  In 
general,  the  policy,  processes,  and 
procedures  of  awarding  contracts  on  factors 
other  than  cost  can  be  termed  Best  Value  (or 


Greatest  Value).  More  specifically,  the  Best 
Value  concept  requires  the  source  selection 
authority  to  examine  the  strengths  or 
advantages  of  each  proposal  to  determine  if 
the  increased  costs  are  of  sufficient  value  to 
the  Government  to  justify  an  award  to  other 
than  the  lowest  responsive  and  responsible 
offer. 


367 


A  METHODOLOGY  FOR  ESTIMATING  BEST  VALUE 


Janine  L.  Faris 
Joan  S.  Lovelace 
The  MITRE  Corporation 


The  goal  of  the  United  States  Government's 
federal  procurement  process  traditionally 
has  been  to  acquire  products  that  meet  the 
minimum  needs  of  the  Government  for  the 
lowest  price  or  lowest  total  cost  to  the 
Government.  However,  the  Federal 
Acquisition  Regulation  (FAR)  also 
recognizes  that: 

While  the  lowest  price  or  lowest  total  cost  to 
the  Government  is  properly  the  deciding 
factor  in  many  source  selections,  in  some 
acquisitions,  the  Government  may  select  the 
source  whose  proposal  offers  the  greatest 
value  (emphasis  added)  to  the  Government 
in  terms  of  performance  and  other  factors  .  . 
FAR  15.605  (c) 

This  is  most  often  the  case  in  negotiated 
procurements  where  the  technical  design  or 
development  approach  may  be  important  to 
the  success  of  the  development  effort.  It  is 
also  increasingly  the  case  in  service 
contracts  where  past  performance  on  similar 
contracts  is  considered.  At  present,  the  term 
"Best  Value",  or  "Greatest  Value"  has  no 
statutory  or  regulatory  definition.  In 
general,  the  policy,  processes,  and 
procedures  of  awarding  contracts  on  factors 
other  than  cost  can  be  termed  Best  Value  (or 
Greatest  Value).  More  specifically,  the  Best 
Value  concept  requires  the  source  selection 
authority  to  examine  the  strengths  or 
advantages  of  each  proposal  to  determine  if 
the  increased  costs  are  of  sufficient  value  to 
the  Government  to  justify  an  award  to  other 
than  the  lowest  responsive  and  responsible 
offer. 


The  policies  and  procedures  pertaining  to 
Best  Value  source  selections  are  delineated 
in  FAR  Part  15  and  the  Federal  Information 
Resources  Management  Regulation 
(FIRMR)  Subpart  201-39.15.  There  is  not  a 
great  deal  of  additional  literature  on  Best 
Value  Analysis.  One  source  is  the  General 
Services  Administration's  (GSA)  document 
on  Best  Value  entitled  Source  Selection: 
Greatest  Value  Approach  (Federal 
Information  Processing  (FIPS)  Resources 
Document  #KMP-92-5-P).  Of  course,  this 
guidance  applies  only  to  Automated  Data 
Processing  (ADP)  procurements  under  the 
GSA  purview.  The  emphasis  of  this 
particular  publication  is  on  how  to  structure 
the  solicitation,  including  how  to  develop 
the  source  selection  plan  and  the  evaluation 
factors.  Although  the  document  describes  a 
four-step  trade-off  methodology  for  the 
cost/technical  trade-off,  as  one  possible 
methodology  for  the  Best  Value  Analysis 
process,  the  guidance  is  brief  and  very  high 
level.  The  document  states  that  there  is  no 
"magic  formula"  in  performing  a 
cost/technical  trade-off,  and  that  whatever 
methodology  is  used  must  be  well- 
documented  with  relevant  facts  and 
supporting  rationale.  It  also  notes  that  non- 
quantifiable  factors  should  also  be  used  in 
determining  the  greatest  value  to  the 
Government. 

In  the  absence  of  statute  or  regulation,  the 
authority  for  how  to  conduct  a  Best  Value 
analysis  must  be  extracted  from  case  law. 
Most  case  law  stems  from  protests  heard 
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before  the  General  Services  Board  of 
Contract  Appeals  (GSBCA),  the  most 
significant  of  these  being  the  Oakcreek 
Funding  (1991)  case,  the  Army's  Reserve 
Component  Automation  System  (RCAS) 
December  1991  case,  and  the  two  Treasury 
Multiuser  Acquisition  Contract  (TMAC) 
decisions.  The  outcomes  of  these  cases  have 
called  for  a  more  rigorous  analysis  when 
award  is  being  made  to  a  technically 
superior  offer  with  a  significantly  higher 
cost  (i.e.,  not  the  lowest-priced  proposal). 
This  has  been  reinforced  by  General 
Accounting  Office  Comptroller  General- 
adjudicated-cases  where  there  was  concern 
as  to  whether  the  technical  superiority  was 
worth  the  cost  premium.  In  calling  for  a 
more  rigorous  analysis,  the  appeal  boards 
have  shown  no  preference  for  descriptive, 
adjectival,  or  point  scoring  evaluation 
systems.  There  has  been  no  objection  to  the 
Government  making  a  cost  adjustment  to  the 
proposal  to  reflect  cost  realism  (based  on  an 
assessment  of  cost  risk),  or  projecting  life 
cycle  costs  to  the  Government. 

Although  quantification  to  assess  the  impact 
of  technical  discriminators  (e.g.,  mission 
effectiveness,  productivity,  training 
effectiveness)  is  accepted,  the  adjudicating 
bodies  have  also  recognized  that  non- 
quantifiable  factors  are  likewise  important. 
Of  some  concern  to  the  cost  estimating  and 
analysis  community  is  the  fact  that 
acceptable  methodologies  for  Best  Value 
Analysis  appear  to  be  increasingly 
determined  outside  the  community  that 
traditionally  performs  value  analysis.  This 
paper  is  an  attempt  to  inject  our  community 
into  the  debate  on  what  constitutes  an 
appropriate  analysis  for  a  Best  Value 
contract  award.  Though  this  paper  is  written 
from  the  perspective  of  someone  serving  on 
a  source  selection  team  and  responsible  for 
performing  the  cost/technical  trade-off;  cost 
analysts  who  are  involved  in  source 


selections,  either  from  the  proposal 
preparation  perspective,  or  the  source 
selection  team  perspective,  can  benefit  from 
an  examination  of  this  issue. 

Planning  for  Best  Value 

A  Best  Value  approach  to  source  selection 
requires  additional  preparation  and  notice  to 
potential  offerors  about  the  evaluation 
methodology.  This  planning  is  necessary  to 
ensure  adequate  competition  and  a  level 
playing  field  and  to  mitigate  the  threat  of  a 
protest.  The  following  considerations,  some 
of  which  are 

discussed  in  detail  below,  are  important  for 
planning  for  Best  Value  evaluation  and 
analysis. 

•  Plan  in  advance  how  the  Best  Value 
Analysis  will  be  conducted.  Since  there 
is  no  generally-accepted  method  of 
performing  a  Best  Value  Analysis,  a 
systematic  methodology  starting  with 
the  Acquisition  Plan  and  coupled  with 
detailed  documentation  of  the  process 
and  logic  utilized  by  the  evaluation 
teams,  will  enhance  the  probability  of 
avoiding  a  protest. 

•  Identify  potential  discriminators  in 
advance  and  build  the  evaluation  factors 
around  them.  Discriminators  must  be 
linked  to  an  evaluation  factor,  even  if 
indirectly.  If  a  discriminator  cannot  be 
linked  to  an  evaluation  factor,  it  cannot 
be  considered.  Similarly,  if  one 
approach  (e.g.,  distributed  processing)  is 
viewed  as  superior  before  the  solicitation 
is  released,  it  must  be  identified  as  a 
requirement  rather  than  a  discriminator. 

•  Anticipate  how  benefits  will  be 
measured  and  collect  the  baseline  data 
needed  for  analysis.  If,  for  example, 
improved  productivity  is  one  of  the 
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anticipated  benefits,  you  will  need  to 
know  the  current  number  of  staff  and  the 
average  output.  If  these  data  are  not 
collected  in  advance,  it  will  lengthen  the 
time  for  the  Best  Value  Analysis  and 
may  raise  questions  regarding  the 
efficacy  of  the  proposal  evaluation 
process. 

•  Determine  how  you  will  perform  the 
risk  analysis  and  how  the  results  will  be 
factored  into  the  overall  evaluation. 
Again,  if  some  technical  risks  can  be 
anticipated,  determine  how  to  translate 
that  risk  into  a  cost. 

At  present,  the  courts  have  upheld  the 
Government's  application  of  a  Best  Value 
Analysis  when  both  the  approach  and  the 
results  have  been  documented  and 
reasonably  support  the  decision  of  the 
Source  Selection  Authority.  The  approach 
described  below  offers  a  starting  point  for 
the  development  of  a  best  value  methodolo¬ 
gy- 

AN  APPROACH  TO  ESTIMATING 
BEST  VALUE 

In  the  absence  of  a  generally-accepted  way 
to  perform  a  Best  Value  Analysis,  this  paper 
offers  a  particular  methodology  as  one  of  the 
ways  in  which  such  an  analysis  can  be 
performed.  This  methodology  consists  of 
the  following  four  steps: 

Step  One:  Cost  Realism  Analysis 

Step  Two:  Quantification  of  Risk 

Step  Three:  Assessment  and  Quantification 
of  Value 

Step  Four:  Cost/Technical  Trade-Off 
Critical  to  the  process  is  the  development  of 


clear  and  cogent  supporting  documentation 
and  rationale,  which  will  be  an  integral  part 
of  the  completed  analysis.  The  Best  Value 
Analysis  will  be  a  key  element  of  the 
evidentiary  documentation  should  there  be  a 
protest. 

Step  One:  Cost  Realism  Analysis 

The  objective  of  the  cost  realism  analysis  is 
to  calculate  the  realistic  cost  to  the  Govern¬ 
ment,  excluding  quantification  of  risk, 
which  is  addressed  later  in  Step  Two.  The 
calculation  of  realistic  costs  normally 
includes  both  the  cost  of  the  acquisition  as 
well  as  operations  and  maintenance  (O&M) 
costs,  or  life  cycle  cost  (LCC).  The  O&M 
cost  may  be  outside  the  scope  of  the 
procurement,  but  is  usually  necessary  to 
develop  in  order  to  accurately  perform  a 
Best  Value  Analysis.  (A  Best  Value 
procurement  for  ADP  under  the  GSA 
purview  requires  that  O&M  costs  be 
evaluated.)  The  inclusion  of  at  least  some 
elements  of  LCC  must  be  included  in  the 
evaluation  criteria  if  the  Best  Value  Analysis 
plans  to  look  at  improved  productivity  as 
projected  cost  savings  from  the  new 
systems. 

After  examining  each  offeror's  cost  proposal 
for  compliance  with  the  proposal  preparation 
instructions  as  delineated  in  Section  L 
(Instructions  to  Offerors)  of  the  solicitation, 
and  the  schedule  as  delineated  in  Section  B 
of  the  solicitation;  the  analyst  must  carefully 
examine  each  offeror's  Basis  of  Estimate  for 
realism,  adequate  substantiation,  and  logic, 
including  the  following: 

•  Staffing  profile 

•  Labor  Rates 
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•  Bill  of  Materials  (If  extensive, 

concentrate  analysis  on  high  value  and 
numerous  quantity  items) 

•  Other  Direct  Costs  (ODCs) 

•  Profit/Fee  (Should  be  consistent  with 
contract  type,  and  not  exceed  statutory 
limits) 

As  part  of  this  process  the  analyst  should 

also  examine  the  following: 

•  Compare  the  key  cost  elements  and 

Basis  of  Estimate  with  industry 

standards  and  practices 

•  Compare  the  Basis  of  Estimate  with  past 
projects/programs 

•  Obtain  technical  evaluation  team  input 
on  the  realism  of  the  proposed  technical 
labor  effort  given  the  technical  approach 


•  Examine  estimates  of  O&M  cost,  if 
required  with  the  proposal  submittal 

The  above-described  process  results  in  the 
development  of  an  Independent  Cost 
Estimate  of  all  or  portions  of  each  offeror's 
proposal.  It  is  critical  to  note  that  any  issues 
of  cost  realism  must  be  discussed  with  the 
offerors,  either  through  the  Clarification 
Request/Deficiency  Report  process,  or 
during  actual  discussions.  The  Independent 
Cost  Estimate  will  generally  be  developed 
using  a  combination  of  estimating 
methodologies  —  parametric,  engineering, 
and  analogy,  and  will  include  an  estimate  of 
O&M  costs.  Following  the  submittal  of 
Best  and  Final  Offers  (BAFOs),  the 
Independent  Cost  Estimate  should  be 
reassessed  and,  if  necessary,  adjusted. 

The  following  table  illustrates  Step  One  of  a 
Best  Value  Analysis  in  a  hypothetical 
procurement: 


Table  1.0.  Step  One  of  a  Best  Value  Analysis  -  Development  of  the  Independent  Cost 

Estimate  Cost  in  Million  Dollars 


|  Price 

Offeror  A 

Offeror  B 

Offeror  C 

Offeror  D 

$55.0 

$59.5 

$65.0 

$50.0 

Cost  Realism 

Independent 

Estimate 

5.0 

4.5 

1.5 

14.2 

Subtotal 

$60.0 

$64.0 

$66.5 

$64.2 

Note:  For  illustrative  purposes  only;  not  derived  from  project  data. 


The  Cost  Realism  Analysis  addresses  Step  Two:  Quantification  of  Risk 
estimating  errors,  incomplete  costs,  and 
unrealistic  costs  (such  as  under  staffing). 
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Concurrent  with  the  cost  proposal  evaluation 
process,  the  technical  team  and  the 
management  team  will  evaluate  the 


respective  technical  and  management 
proposals.  Generally  this  evaluation  is 
conducted  independent  of  the  cost  proposal 
evaluations.  Similar  to  the  cost  proposal 
evaluation,  the  technical  and  management 
evaluation  teams  do  the  following: 

•  examine  each  proposal  in  relation  to  the 
evaluation  factors  and  significant 
subfactors  (as  reflected  in  the  source 
selection  plan  and  Section  M  of  the 
solicitation) 

•  measure  each  proposal  against  the 
evaluation  standards  and  assign  a  rating 

•  identify  proposal  ambiguities  and 
inadequate  proposal  substantiation  and 
have  the  Contracting  Officer  obtain 
clarifications  and  additional  information 

•  identify  and  document  the  strengths, 
weaknesses,  and  risks  of  the  competing 
proposals 

•  identifydeficiencies/ unacceptable 
proposals 

•  if  possible,  arrive  at  a  consensus  of  the 
team  on  the  final  rating  of  each  technical 
proposal  (if  consensus  is  not  possible, 
thoroughly  document  dissenting  opinions) 

•  work  with  the  cost  evaluation  team  to 
establish  the  competitive  range 


•  hold  meaningful  discussions/negotiations 
with  all  of  the  offerors  in  the  competitive 
range 

The  identification  and  documentation  of  the 
strengths,  weaknesses,  and  risks  of  each 
proposal  lays  the  groundwork  for  the  risk 
assessment  and  cost/technical  trade-off  steps 
of  the  Best  Value  Analysis,  because  it 
determines  the  significant  areas  of 
difference,  or  discriminators,  among  the 
proposals.  Following  BAFO  submittal,  the 
technical  and  management  evaluation  teams 
share  the  results  of  their  evaluations  with  the 
cost  team.  At  this  point  the  combination  of 
ratings  are  determined.  If  all  proposals  are 
determined  to  be  technically  equal,  then 
award  should  be  made  to  that  offeror  with 
the  lowest  cost  proposal.  If  the  proposal 
with  the  highest  technical  merit  is  also  the 
lowest  cost  proposal,  then  award  should  be 
made  to  that  offeror.  However,  the  source 
selection  is  usually  not  that  simple.  For 
example,  the  lowest  priced  proposal  may 
have  unacceptable  or  high  technical  risk. 
Conversely,  the  highest  technical  proposal 
may  have  the  highest  price;  however,  further 
analysis  may  show  that  the  highest  price 
may  be  warranted  by  low  risk  and  superior 
management.  This  paper  assumes  a  scenario 
in  which  the  technical  proposals  are  not 
technically  equal  in  merit.  Table  2.0  is  an 
illustrative  representation  of  the  initial 
results  of  all  the  team  evaluations. 
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Table  2.0.  Initial  Reports  from  Technical,  Management,  and  Cost  Evaluation  Teams  Cost 

in  Million  Dollars 


Price 

Offeror  A 

Offeror  B 

Offeror  C 

Offeror  D 

Technical  Team 

Rating 

Fourth 

Second 

First 

Third 

Qualitative  Risk 
Assessment 

High 

Medium 

Superior 

Acceptable 

Management  Team 

Rating 

Acceptable 

Superior 

Superior 

Acceptable 

$55.0 

$59.5 

$65.0 

$50.0 

$60.0 

$66.5 

$64.2 

Note:  For  illustrative  purposes  only;  not  derived  from  project  data. 


In  this  portion  of  the  analysis,  the  cost 
evaluation  team  must  quantify  the  technical 
and  management  risks  which  have  been 
identified  during  those  respective 
evaluations.  To  accomplish  this 
quantification,  the  cost  evaluation  team  will 
work  closely  with  the  technical  and 
management  evaluation  teams.  This  quan¬ 
tification  effort  translates  the  technology  and 
business  risks  associated  with  each  offeror's 
proposal  into  cost.  Each  quantifiable  risk 
has  a  range  of  cost,  with  probabilities 
associated  with  each  point  estimate  falling 
within  this  range.  Typical  high-risk  areas 
include,  but  are  not  limited  to,  the  following: 

•  Maturity  of  technology  (state-of-the-art) 

•  Reliability,  availability,  maintainability 
requirements 

Levels  and  types  of  integration 

•  Rights  to  technical  data 

•  Business  management  risks  (e.g., 
performance  on  past  programs) 


•  Schedule  risks 

•  Multinational  teams 

Not  all  risks  are  quantifiable.  These 
residual,  qualitative  risks  (e.g.,  ease  of  use, 
dependency  on  key  individuals,  etc.)  must 
also  be  addressed  and  included  in  the  Best 
Value  Analysis.  The  result  of  the  risk  as¬ 
sessment  is  the  expansion  of  the  point 
estimate  into  a  range  estimate  with  a 
corresponding  probability  distribution.  A 
point  estimate  with  an  associated  probability 
of  completion  at  the  estimated  cost  and 
schedule,  which  is  acceptable  to  the  agency, 
can  then  be  selected.  The  selected 
probability  is  dependent  upon  contract  type 
and  how  much  risk  the  agency  is  willing  to 
assume.  For  example,  if  the  contract  will  be 
Cost  Plus  Fixed  Fee,  then  the  analyst  may 
want  to  select  a  value  for  the  estimate  which 
has  a  higher  probability  of  success,  such  as 
an  80%  probability,  than  if  the  contract  is 
one  of  the  Fixed  Price  type  of  contracts.  In 
the  latter  case  a  lower  probability  of  success 
may  be  warranted,  such  as  a  50% 
probability.  The  combination  of  the 
independent  cost  estimates  and  the  risk  as- 


373 


sessments  results  in  a  most  probable  cost  to 
the  Government.  Table  3.0  illustrates  the 


additive  nature  of  the  Best  Value  Analysis. 


Table  3.0.  Step  Two  -  Quantification  of  Risk 
Cost  in  Million  Dollars 


1  Price 

Offeror  A 

Offeror  B 

Offeror  C 

Offeror  D 

$55.0 

$59.5 

$65.0 

$50.0 

Cost  Realism  In¬ 
dependent  Es¬ 

timate 

5.0 

■ 

1.5 

14.2 

Tech¬ 
nical/Manage¬ 
ment  Risk 
Quantification 
Range 

5.0  -  6.3 

i 

1.5-2.75 

.5-1.2 

2.5 -3.2 

80%  Probability 
of  Success 

6.0 

1.0 

3.0 

Most  Probable 

$66.0 

$66.5 

$67.5 

$67.2 

Cost  to 

the  Government 

Note:  For  illustrative  purposes  only;  not  derived  from  project  data. 


At  the  completion  of  this  step  in  the  Best 
Value  Analysis,  the  cost,  schedule, 
technical,  and  management  risks  have  been 
identified  and  quantified  where  possible. 

Step  Three:  Assessment  and 

Quantification  of  Value 

The  next  step  in  the  Best  Value  Analysis  is 
for  the  cost  analyst  to  assess  and  quantify 
value,  to  the  maximum  extent  possible.  This 
assessment  is  derived  from  the  strengths  and 
weaknesses  which  have  been  identified 
earlier  in  the  evaluation  process.  These 


strengths  and  weaknesses  are  linked  to  the 
evaluation  factors/significant  subfactors,  and 
have  been  defined  in  regard  to  each 
evaluation  factor/significant  subfactor's 
respective  evaluation  standard,  ensuring  a 
good  audit  trail  and  minimization  of  bias. 
Examples  of  potential  value  element 
include: 

•  additional  costs  for  renewal  of  licenses 

•  a  larger  footprint  for  a  particular 
offeror's  system 

•  early  delivery 
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•  better  management  structure 

•  more  experienced  personnel 

•  greater  adherence  to  open  systems 
standards 

•  better  software  development  practices 

•  superior  user  training  package 

•  friendlier  user  interface 


•  corporate  history  of  excellent 
performance 

If  weaknesses  are  perceived,  then  the  value 
will  be  reduced.  Those  value  elements 
which  cannot  be  quantified  are  set  aside  and 
later  factored  into  the  evaluation  process. 
Table  4.0  presents  the  results  of  a 
hypothetical  quantification  ofvalue. 


Table  4.0.  Step  Three  -  Assessment  and  Quantification  of  Value  Cost 

in  Million  Dollars 


I  Price 

Offeror  A 

Offeror  B 

Offeror  C 

Offeror  D 

— 

$2.5 

— 

1.5 

Friendlier  User 
Interface 

l 

l 

Superior  User 

Training 

— 

— 

3.0 

“ 

"Lights  Out" 

.5 

2.5 

3.0 

2.0 

Earlier 

Obsolescence 

: - 

— 

2.5 

1.0 

Value 

$0.5 

$8.0 

$8.5 

$4.5 

Note:  For  illustrative  purposes  only;  not  derived  from  project  data. 


Step  Four:  Cost/Technical  Trade-off 

The  next  step  in  the  Best  Value  Analysis  is 
to  perform  the  Cost/Technical  Trade-off  by 
examining  the  most  probable  cost  in  relation 
to  the  quantified  values  (which  have  been 
developed  from  the  technical  and 
management  assessment  of  strengths  and 
weaknesses).  A  net  value  for  each  proposal 
is  developed  by  subtracting  the  marginal 
benefits  from  the  most  probable  cost.  The 


net  value  is  a  metric  used  to  examine 
comparative  adjusted  cost  given  benefits,  or 
"value"  to  the  Government.  Net  value  does 
not  represent  the  estimated  cost  that  the 
Government  may  pay  "out-of-pocket"  for 
this  acquisition  —  that  value  is  represented 
by  the  Most  Probable  Cost.  The  proposals 
are  then  ranked  given  the  results  of  the  net 
value.  Table  5.0  presents  the  net  value 
result  of  the  cost/technical  trade-off. 
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Table  5.0.  Net  Value  By  Offeror 
Cost  in  Million  Dollars 


Cost  Element 

Offeror 

A 

Offeror 

B 

Offeror 

C 

Offeror 

D 

Most  Probable 
Cost 

$66.0 

$66.5 

$67.5 

$67.2 

Value 

.5 

8.0 

8.5 

4.5 

Net  Value 

$65.5 

$58.5 

$59.0 

$62.7 

Note:  For  Illustrative  purposes  only;  not  derived  from  project  data 


The  results  in  Table  5.0  indicate  that  Offeror 
B  is  the  best  quantified  value  at  this  point  in 
the  analysis. 

Table  6.0  presents  a  history  of  how  the 
relative  rankings  have  changed  at  each  step 
in  this  Best  Value  Analysis,  with  cost 


rankings  delineated  as  "First"  being  the 
lowest  cost,  and  "Fourth"  being  the  highest 
cost.  Likewise,  technical  rankings  are 
delineated  as  "First"  being  the  highest 
technically-ranked  proposal,  and  "Fourth" 
being  the  lowest  technically-ranked 
proposal. 


Table  6.0.  Relative  Rankings  For  Each  Step  of  the  Best  Value  Analysis 
Cost  in  1994  Million  Dollars 


Best  Value 

Analysis  Step 

Offeror  A 

Offeror  B 

Offeror  C 

Offeror  D 

Proposed  Price 

Second 

Third 

Fourth 

First 

$55.0 

$59.5 

$65.0 

$50.0 

Cost  Realism 

First 

Second 

Fourth 

Third 

Independent 

Estimate 

$60.0 

$64.0 

$66.5 

$64.2 

Technical  Team 
Rating 

Fourth 

Second 

First 

Third 

First 

Second 

Fourth 

Third 

Most  Probable 

Cost 

$66.0 

$66.5 

$67.5 

$67.2 

Fourth 

First 

Second 

Third 

Net  Value 

$65.0 

$58.5 

$59.0 

$62.7 

Note:  For  Illustrated  purposes  only;  not  derived  from  project  data 


The  recap  of  rankings  in  Table  6.0  illustrates 
how  the  value  of  each  offeror's  proposal,  and 
the  relative  rankings,  changed  during  the 


Best  Value  Analysis  process.  When  the  cost 
evaluation  team  calculates  Net  Value,  all  of 
the  quantifiable  costs,  risks,  and  value  have 
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been  assessed  and  factored  into  the  analysis. 
The  results  in  Table  6.0  indicate  that  at  this 
step  in  the  Best  Value  Analysis,  the  second 
highest  technically-ranked  offeror.  Offeror 
B,  is  the  best  quantified  value.  The  last  step 
in  the  Best  Value  Analysis  is  to  consider  the 
non-quantified  risks  and  values  of  each 
offeror's  proposal.  Once  the  qualitative  risks 
and  values  have  been  factored  into  the 
evaluation,  then  the  Best  Value  to  the 
Government  can  be  determined,  and  a 
source  selection  decision  can  be  made. 

Bringing  it  all  Together 

The  final  step  is  to  present  to  the  Source 
Selection  Advisory  Council  (or  SSA)  the 
results  of  the  quantitative  analysis  and  the 
qualitative  assessment  and,  depending  on  the 
preferences  of  the  SSA,  a  recommendation 
for  award.  Before  the  final  ranking,  the 
quantitative  analysis  should  be  reviewed 
closely.  In  particular,  the  range  of  probable 
costs  associated  with  the  risk  assessment 
should  be  examined.  Where  the  offerors' 
most  probable  cost  ranges  overlap,  it  might 
be  advisable  to  perform  a  sensitivity 
analysis.  The  non-quantified  or  residual 
risks  should  be  revisited  to  ascertain  whether 
any  were  serious  enough  to  influence  the 
ranking;  for  example,  an  adjectival 
description  of  high,  medium  or  low  could 
be  assigned  to  the  residual  risks.  These 
could  be  translated  into  a  percentage  of 
additional  cost. 

In  a  similar  fashion,  the  non-quantitative 
values  must  be  assessed. 

Throughout  this  approach,  the  attempt  has 
been  made  to  quantify  what  can  be 
quantified  objectively  and  to  retain  the  non- 
quantifiable  elements  for  the  complete 
analysis.  The  ability  to  quantify  a  particular 
risk  or  a  benefit  is  often  a  function  of  the 
time  and  data  available.  To  the  extent  that 


improvements  in  productivity  or  operating 
costs  can  be  anticipated,  those  persons 
responsible  for  delineating  the  evaluation 
process  should  also  be  responsible  for 
collecting  the  data  and  establishing  the 
baseline  in  advance  of  the  evaluation; 
however,  some  offerors'  benefits  will  not  be 
foreseeable  and  in  other  cases  the  data  may 
be  too  time-consuming  or  expensive  to 
collect.  Subjective  estimates  of  value  can 
never  be  completely  eliminated  from  the 
process.  Also,  it  is  worth  emphasizing  here 
that  even  the  most  highly  quantified  analysis 
does  not  result  in  "an  answer."  The  purpose 
of  the  Best  Value  Analysis  is  to  support  the 
decision-maker,  not  make  the  decision. 

Issues 

There  are  several  issues  associated  with  a 
Best  Value  Analysis  that  remain  unresolved 
for  the  authors. 

1.  Should  the  analysis  be  conducted  at 
the  element  rather  than  the  aggregate 
level?  There  is  an  underlying  assumption 
that  the  difference  in  cost  between  the  most 
probable  cost  of  two  proposals  can  be 
attributable  to  some  difference  in  either  the 
product  itself  or  the  process  of  developing  or 
providing  the  product.  Most  of  the  analyses 
performed  to  date,  as  in  the  example  above, 
analyze  both  the  costs  and  benefits  at  the 
aggregate  level.  This  masks  the  relationship 
between  the  cost  differential  and  the  benefit 
for  any  individual  element.  At  the  element 
level,  there  is  not  necessarily  a  direct 
correlation  between  the  increased  value  of 
an  element  and  an  increase  in  cost.  In  fact, 
at  the  element  level,  an  increase  in  value  for 
superior  software  development  methodology 
may  result  in  lower  costs.  If  a  portion  of  the 
proposal's  higher  overall  costs  cannot  be 
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attributable  to  the  value  of  an  element,  it  is 
unclear  how  that  case  should  be  considered 
in  the  analysis. 

2.  Should  the  Best  Value  Analysis  weight 
the  benefits  in  accordance  with  the 
weights  assigned  to  the  evaluation  factors 
in  the  Source  Selection  Plan?  As  noted 
above,  all  the  offerors  considered  in  the  Best 
Value  Analysis  meet  the  Government’s 
requirements.  The  Best  Value  Analysis,  for 
the  most  part,  deals  with  marginal  value. 
Frequently  the  discriminators  relate  back  to 
relatively  low-weighted  evaluation  factors. 
However,  where  there  is  a  difference  in  the 
factor  weights,  the  relative  importance  has 
not  been  adequately  addressed  in  the  Best 
Value  Analysis.  To  some  extent  the  "value" 
estimated  for  each  discriminator  considers 
the  importance  of  that  feature,  but  only 
imperfectly.  This  issue  has  already  been 
raised  as  part  of  the  TMAC  protest  and 
although  the  Government's  decision  was 
sustained,  it  should  not  be  considered 
settled. 

In  summary,  the  methodology  described 
above  takes  only  a  few  steps  beyond  the 
FIPS  guidelines.  It  is  imperative  that  the 
cost  estimating  and  economic  analysis 
community  become  involved  in  the 
development  of  Best  Value  methodologies  — 
it  should  not  be  left  to  lawyers  and  boards  of 
contract  appeals  to  determine. 
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ABSTRACT 

Assessment  of  performance  is  a  key 
issue  in  the  DOD.  Commanders  of  all  types 
of  organizations  are  constantly  searching  for 
metrics  to  determine  if  their  organizations 
are  improving.  We  address  ways  of 
measuring  improvement  in  this  paper.  This 
issue  has  become  increasingly  significant  in 
light  of  the  National  Performance  Review 
(NPR)  (Gore,  [9]).  In  this  study,  we 
examine  three  USAF  policy  alternatives:  (1) 
privatization  of  USAF  functions,  (2) 
promotion  of  competition  between  USAF 
facilities  and  other  government  entities  or 
civilian  contractors,  and  (3)  continuous 
improvement  of  USAF  processes.  We  argue 
that  these  initiatives  should  support  the  goals 
of  increasing  efficiency  while  maintaining 
combat  capability.  To  determine  which 
policy  initiative  is  appropriate,  we  suggest 
activity  based  costing  (ABC)  and  financial 
analysis  as  means  of  measuring  costs  and 
assessing  performance. 

INTRODUCTION 

The  American  defense  budget  has 
been  shrinking  in  real  terms  since  the  mid 
1980s.  From  1985  through  1997,  the  DOD 
projects  overall  defense  budget  cuts  to  equal 
42%,  personnel  cuts  of  30%,  overseas  base 
closings  of  35%,  and  US.  base  closings  of 
15%  (DOD  Comptroller,  [4]).  Despite  these 
cuts,  the  requirements  to  support  critical 


national  interests  in  various  regions  of  the 
world  remain.  With  shrinking  budgets  and 
steadily  growing  requirements,  DOD  is 
focusing  on  initiatives  to  increase  efficiency 
of  operations.  The  Air  Force,  for  example,  is 
experimenting  with  a  new  composite  wing 
model  that  consolidates  operations  and 
reduces  the  cost  of  administrative  and 
command  redundancies  (McPeak,  [15]). 
Despite  these  attempts,  numerous  factors 
such  as  the  DOD's  complex  organizational 
structure,  lack  of  incentives  to  improve 
operations,  and  ever  shifting  requirements 
and  direction  have  hampered  improvement 
efforts.  Yet  many  within  DOD  believe  that 
dramatically  improved  efficiencies  are 
possible  without  decreases  in  combat 
capability. 

These  efficiencies  can  come  from  two 
sources.  First,  some  DOD  functions  may  be 
better  performed  by  the  private  sector. 
Second,  some  DOD  processes  can  be 
improved.  Minimizing  overall  costs  while 
maintaining  combat  capability  is  the  goal. 
DOD  policy  has  always  been  directed  at 
getting  the  most  goods  and  services  for  the 
least  amount  of  money.  However,  finding 
new  and  innovative  ways  to  reduce  costs  is 
particularly  important  during  the  current 
downsizing  period. 

While  this  goal  of  improved 
efficiency  is  important,  there  are 
impediments.  They  include  a  lack  of  political 
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will.  Prager  [18]  argues  that  the  public 
sector  is  inefficient  because  of  "...a  lack  of 
political  will  to  establish  efficiency  as  a  high- 
level  priority  of  government  operations"  (p. 
180).  He  goes  on  to  say  that  policy  makers 
de-emphasize  efficiency  as  a  goal  by  not 
providing  management  with  the  flexibility  to 
pursue  efficiency  goals  and  because  "...the 
incentive  structure  of  the  public  sector  either 
is  neutral  toward  or  even  discourages  cost 
savings"  (p.  180).  Kelman  [13]  suggests  that 
the  lack  of  competition,  autonomous  choice, 
profit  motivation;  and  timely  procurement 
source  selection  decisions  undermine 
government  efficiency.  Values  like  an 
equitable  distribution  of  government 
contracts  for  procured  goods  are  sometimes 
competing  goals  as  well.  Lastly,  Meier  [16] 
argues  that  efficiency  within  many  public 
organizations  is  not  even  a  relevant  goal, 
"The  goal  of  government  agencies  is 
universal  service  rather  than  efficient  service" 
(P-7). 

Other  goals  and  objectives  at  stake 
when  addressing  efficiency  come  from  the 
defense  industry,  DOD  employees,  and 
Congress.  Defense  industry  management  has 
the  goal  of  maximizing  shareholder  wealth. 
Wringing  efficiencies  out  of  DOD  may  come 
partly  at  contractor  expense.  DOD 
personnel  want  to  maintain  their  jobs  and 
advancement  potential  within  their 
organizations.  Congress  also  is  a  major 
policy  actor  that  fears  losing  control  if 
various  streamlining  initiatives  are 
accomplished. 

EFFICIENCY  AND  COMBAT 
CAPABILITY 

The  dilemma  facing  the  USAF  is  that 
it  must  be  both  efficient  and  combat  capable. 
A  policy  design  must  value  both  goals. 
Combat  capability,  training  and  equipment, 


and  the  mental  state  of  the  fighting  force  may 
conflict  with  policies  that  improve 
efficiencies.  One  example  is  deciding  to  buy 
a  new  weapon  system  rather  than  upgrading 
an  older  weapon  system.  The  way  to 
decrease  costs  may  be  to  simply  upgrade  an 
older  system,  but  this  may  not  be  appropriate 
depending  on  the  enemy’s  threat  level,  state 
of  technology  and  other  factors. 

Incentives  may  be  useful  to  motivate 
individuals  to  perform  with  cost 
minimization  in  mind.  As  Arrow  [1] 
suggests  in  principal-agent  theory,  the  costs 
associated  with  an  agent  pursuing  self- 
interested  behavior  must  be  minimized  by 
instituting  rewards  that  motivate  behavior 
desirable  to  the  principal.  These  rewards 
need  not  be  strictly  financial.  Power, 
prestige  and  public  service  are  other  valid 
rewards  (Prager,  [18]).  Ingram  and 
Schneider  [12]  describe  motivational  tools 
useful  for  public  sector  decision  managers. 
These  tools  include:  authority,  capacity 
building,  symbolic  and  hortatory,  and 
learning  tools. 

Deciding  how  to  reward  public  sector 
employees  is  often  a  difficult  task.  In  some 
areas  where  government  agencies  perform 
services  similar  to  those  offered  by  the 
private  sector,  comparisons  of  efficiency  may 
be  appropriate.  For  example  in  the  USAF, 
health  care,  vehicle  and  aircraft  maintenance, 
and  security  have  similar  counterparts  in  the 
private  sector.  Comparisons  within  these 
activities  may  shed  light  on  how  to  reward 
employees  and  improve  operations.  But 
comparisons  are  difficult  where  the  function 
has  fewer  private  sector  characteristics  (such 
as  combat  operations). 

Before  developing  specific  policy 
alternatives,  we  define  efficiency  and  combat 
capability.  Two  definitions  of  efficiency  are: 
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“getting  the  greatest  aggregate  good  from 
available  resources”  (Weimer  and  Vining, 
[22],  p.  16)  and  “...the  ratio  between  inputs 

(1)  and  outputs  (O)”  (Brewer  and  deLeon, 

[2] ,  p.  335).  Cost  minimization  is  a  more 
precise  assessment  of  the  input  factors  in 
Brewer  and  deLeon’ s  definition. 

Combat  capability  includes  such 
concepts  as:  combat  readiness,  a  high  level 
of  training,  the  right  equipment  to  do  the  job, 
and  a  mentally  prepared  fighting  force. 
Some  definitions  of  combat  readiness  are 
concerned  with  military  utilization  rates  of 
weapon  systems,  or  the  percentage  of 
weapon  systems  that  are  ready  for  war,  or 
simply  how  prepared  a  military  unit  is  for 
combat  as  measured  by  an  inspection  team. 
A  well  trained  and  equipped  fighting  force 
uses  various  training  methods  to  keep  their 
skills  honed.  A  well  equipped  force  has  an 
adequate  level  of  equipment  that  is  reliable 
and  technologically  superior  to  the  enemy. 
A  combat  capable  fighting  force  should  also 
be  mentally  prepared  for  combat,  have  high 
morale  and  camaraderie,  and  a  strong  culture 
that  rewards  achieving  the  desired  mission 
outcomes. 

These  two  ideals— efficiency  and 
combat  capability  often  conflict.  When  this 
happens  policy  makers  must  determine 
whether  efficiency  or  combat  capability  will 
take  precedence. 

POLICY  ALTERNA  TIVES 

We  now  turn  to  a  discussion  of 
policy  alternatives,  using  USAF  examples  to 
illustrate  how  efficiencies  can  be  gained  and 
measured.  The  initiatives  are  (1) 
privatization  of  USAF  functions,  (2) 
promotion  of  competition  between  USAF 
facilities  and  other  government  entities  or 


civilian  contractors,  and  (3)  continuous 
improvement  of  USAF  processes. 

Option  1:  Privatization  of  USAF 

functions 

Privatization  is  “...enlisting  private 
energies  to  improve  the  performance  of  tasks 
that  would  remain  in  some  sense  public” 
(Donahue,  [5],  p.  7).  The  USAF  performs 
many  functions  “in-house”  such  as  building, 
vehicle  and  aircraft  maintenance,  financial 
management,  medical  care,  fire  fighting,  and 
security.  Many  of  these  tasks  could  be 
performed  by  the  private  sector.  Many 
privatization  examples  already  exist  within 
the  USAF.  Most  building  janitorial  functions 
are  performed  by  private  contractors  rather 
than  military  or  civilian  defense  employees. 
Also,  some  maintenance  functions  of  newly 
acquired  defense  hardware  are  accomplished 
in  the  private  sector.  This  option  demands  a 
much  larger  and  wide-scaled  review  of 
USAF  functions  to  privatize  all  activities  that 
would  yield  a  cost  savings  and  would  not 
negatively  impact  wartime  capability. 

Privatization  decisions  must  consider 
combat  capability.  Kettl  [14]  argues  for 
more  privatization,  but  cautions  against 
privatizing  “core  functions.”  At  the  same 
time,  the  USAF  must  guard  its  ability  to 
provide  the  appropriate  surge  capability. 
The  USAF  is  moving  towards  purchasing 
private  sector  products  and  processes  where 
possible.  This  shift  from  militarily  unique 
products  and  services  will  both  increase 
military  access  to  the  national  industrial  base 
and  support  the  growing  trend  towards 
privatization  of  public  sector  services,  both 
in  this  country  and  abroad. 

Given  the  policy  initiative  of 
privatizing,  how  should  the  USAF  manager 
decide  whether  the  efficiency  gained  from 
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privatizing  outweighs  any  possible  impact  on 
combat  capability?  We  suggest  the  use  of 
traditional  private  sector  financial  analysis  to 
compare  public  and  private  sector 
operations.  If  financial  analysis  suggests  that 
the  public  function  performs  much  less 
efficiently  than  the  comparable  private  sector 
function,  then  the  option  of  privatization 
should  be  considered.  This  option  may  be 
particularly  attractive  if  the  privatization 
does  not  impact  combat  capability  and  valid 
private  sector  comparisons  are  available. 

The  continuing  revolution  in 
government  accounting  has  made  possible, 
for  the  first  time,  the  financial  comparisons 
of  public  and  private  sector  functions.  We 
use  a  simple  example  to  illustrate.  Using 
financial  data  compiled  by  the  Defense 
Financial  Accounting  Service,  we  compute 
several  financial  ratios  for  a  large 
government  owned  bookstore  and  compare 
them  to  similar  ratios  of  two  private 
booksellers,  Crown  Books  Inc.  and  Barnes 
and  Noble  Inc.  The  data  for  these 
corporations  is  available  from  Compustat. 

Financial  ratio  analysis  is  a  technique 
used  in  the  private  sector  to  compare  the 
operations  of  firms.  This  technique 
facilitates  the  analysis  of  differently  sized 
organizations  within  the  same  line  of 
business.  Ratios  measure  liquidity,  asset 
management,  and  profitability.  (See 
Harrison  and  Homgren,  [10]  Chapter  15). 
In  this  example,  we  focus  on  asset 
management.  Table  1  shows  the  asset 
management  ratios  a  large  government 
owned  bookstore  relative  to  Crown  Books 
and  Barnes  and  Noble.  The  results  show 
that  asset  turnover  a  large  government 
bookstoreis  much  lower  than  that  of  its 
private  sector  counterparts.  Asset  turnover 
measures  how  well  an  entity  uses  its  assets  to 
generate  sales.  Low  turnover  implies  that 


larger  amounts  of  assets  are  needed  to 
generate  a  given  level  of  sales.  In  this 
particular  case,  a  low  asset  turnover  may 
mean  that  the  entity  keeps  a  lot  of  inventory 
relative  to  its  peers.  This  assertion  is 
supported  by  the  inventory  turnover  findings. 
A  large  government  owned  bookstore 
inventory  turnover  is  much  lower  than  its 
rivals.  Together,  these  two  findings  may 
indicate  that  inventory  policy  at  the  public 
entity  may  be  driving  up  its  costs  relative  to 
its  competitors. 

Other  ratios  in  Table  1  support  this 
conclusion.  A  large  government  owned 
bookstore  has  a  low  sales  to  net  working 
capital  ratio  and  a  very  high  liquidity  ratio. 
Both  are  indications  of  high  levels  of  current 
assets  like  inventory.  The  entity  also  has  low 
accounts  receivable  turnover,  meaning  that  it 
does  not  collect  very  rapidly  for  the  sales 
that  it  does  make.  All  of  these  ratios 
together  point  to  a  need  to  examine  asset 
management  policy. 

Option  2.  Competition  Between  USAF 
and  Other  Government  or  Civilian 
Contractors 

This  initiative  examines  how 
efficiencies  can  be  gained  by  facilitating 
competition  between  USAF  functions  and 
other  government  entities  or  civilian 
contractors.  This  competition  may  cause 
USAF  providers  to  become  more  efficient  or 
lose  work  to  their  competitors.  Prior  to 
several  years  ago,  private  contractors  were 
not  permitted  to  compete  in  many  of  these 
areas.  Furthermore,  competition  between 
government  entities  was  not  possible  because 
of  the  lack  of  prices  for  internally  transferred 
government  goods  and  services.  Transfer 
pricing  regulates  the  cost  of  internally 
manufactured  goods  and  services.  For 
example,  a  Navy  shipyard  may  manufacture 
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its  own  anchors.  Suppose  the  customer,  the 
operational  Navy  command,  had  the  choice 
to  pay  for  the  anchors  through  a  financial 
transfer  of  their  cost  to  the  shipyards  or  had 
the  ability  to  buy  the  anchors  from  a  non- 
Navy  manufacturer.  The  shipyard  workers 
would  have  the  incentive  to  do  it  at  the  least 
cost.  The  Navy  would  be  able  to  make  a 
cost  and  quality  comparison  between 
externally  supplied  anchors  and  the  anchors 
made  by  the  shipyard.  A  decision  could  then 
be  made  to  buy  them  by  selecting  the 
cheapest  supplier  with  the  best  quality 
anchor.  Recognize  that  under  current 
procurement  rules  such  a  choice  is  not 
possible.  The  anchors  would  be  supplied  by 
the  Navy  shipyard.  The  shipyard  would 
build  them  with  funding  from  its  own  budget 
rather  than  from  a  fund  transferred  from  the 
customer,  or  if  contracted  out,  the  lowest  bid 
contractor  would  be  selected. 

Another  example  of  competition  can 
be  found  at  the  USAF  maintenance  facility  at 
Ogden  AFB,  in  Salt  Lake  City,  Utah.  It  has  a 
major  landing  gear  repair  operation  that  is 
well  suited  to  repair  many  types  of  aircraft. 
Competition  between  the  Navy  and  the  Air 
Force  to  make  repairs  on  jet  fighters  like  the 
F-18  or  the  F-16  is  aimed  at  inspiring  repair 
facilities  to  minimize  costs  through  more 
efficient  operations.  To  incentivize  workers, 
financial  rewards  for  efficient  work  could  be 
given.  The  incentives  need  not  concentrate 
solely  on  financial  rewards.  For  example, 
competition  might  be  fostered  through 
"winning"  a  reserved  parking  spot  that  was 
conveniently  located  near  their  crowded 
workplace.  This  could  be  an  incentive  to 
improve  output  or  reduce  costs. 

To  effectively  compete,  credible  and 
relevant  costs  and  prices  must  be 
determined.  Private  sector  corporations  use 
transfer  pricing  as  a  basis  to  determine 


internal  prices  and  costs.  One  way  to 
facilitate  transfer  pricing  in  the  public  sector 
is  through  a  revolving  fund.  A  revolving 
fund  is  a  financial  tool  used  to  transfer 
charges  between  public  organizations. 
Currently,  the  DOD  uses  the  Defense 
Business  Operations  Fund  (DBOF)  as  its 
revolving  fund.  It  was  formed  by  the  1991 
Defense  Authorization  Bill  and  merged  the 
Stock  and  Industrial  Funds  into  one  large 
fund  (The  Defense.  [3]).  As  of  1994,  DBOF 
contains  $85  billion  that  allows  certain 
organizations  to  transfer  the  charges  for 
products  between  them  (DOD  Comptroller, 
[4]).  The  cost  of  non- value  added  activities 
are  also  highlighted  because  visible  prices  are 
attached  to  those  activities. 

This  revolution  is  also  being  carried 
forward  by  the  growing  emphasis  on  ABC 
methods  in  the  accounting  community. 
These  methods  tie  the  costs  of  production  or 
services  more  closely  to  the  activities  that 
drive  the  costs.  (See  Hilton,  [11]  Chapter 
5).  Before  DBOF  and  ABC,  the  costs  of 
many  requested  goods  and  services  were 
"hidden"  to  managers.  With  visible  costs, 
managers  may  decide  whether  a  product  is 
really  needed  before  requesting  it.  If  costs 
are  too  high,  the  manager  or  commander  can 
shop  elsewhere  for  the  product  or  do 
without.  In  a  related  study,  Eldenburg  [6] 
concluded  that  making  costs  visible  to 
physicians  reduced  the  cost  of  health  care. 

The  DBOF,  as  currently  employed, 
has  many  restrictions  that  interfere  with  true 
market-like  incentives.  One  of  the  most 
significant  limitations  is  the  utilization  of 
legislatively  established  rates,  rather  than 
market  prices  or  unit  cost,  as  a  transfer 
pricing  policy  (OPM,  Unit  Cost  Resourcing. 
[17,  21]).  Legislative  controls  on  prices  and 
lack  of  profit  incentives  impede  the  efficient 
allocation  of  resources  within  public 
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organizations.  A  public  organization  that  is 
allowed  to  make,  retain,  spend  and  distribute 
a  profit  as  management  determines  may 
further  improve  the  efficiency  of  these 
organizations. 

Option  3:  Continuous  Improvement  of 
Existing  Operations 

Improving  overall  USAF  efficiency  is 
an  important  organizational  goal,  especially 
during  military  downsizing.  Open 
competition  and  privatization  are  alternatives 
that  may  help  to  improve  efficiency. 
However,  these  policy  alternatives  may  be 
inappropriate  for  many  USAF  functions. 
Combat  capability  demands  that  some  USAF 
functions  be  done  in  house.  The  issue  thus 
becomes  how  to  do  a  more  efficient  job  on 
these  functions.  The  historical  focus  of 
public  fiscal  activities  has  been  on 
expenditures  and  “zeroing  out”  one’s 
budget,  rather  than  on  reducing  costs 
(Gansler,  [7]).  As  Donahue  [5]  argues, 
cumbersome  rules  and  procedures  associated 
with  public  procurements  contribute  to  the 
lack  of  efficiency. 

One  can  get  a  rough  measure  of 
overall  efficiency  by  allocating  the  total 
USAF  budget  to  a  unit  of  output  like  aircraft 
hours  flown  or  number  of  air  crews.  For 
example,  with  a  $85  billion  budget  in  1993,  a 
5%  savings  would  be  about  $4.25  billion.  If 
the  USAF  can  keep  its  annual  flying  hours 
constant  at  2,905,000,  then  some  efficiencies 
may  occur.  In  this  example,  if  the  savings 
could  be  achieved,  then  policy  makers  could 
use  the  savings  for  other  valued 
opportunities  within  the  DOD  or  other 
federal  programs. 

Knowledge  of  total  costs  as  revealed 
through  modern  management  information 
systems  and  innovative  accounting  practices 


can  help  managers  at  lower  organizational 
levels  to  accurately  track  the  true  cost  of 
producing  output.  This  is  an  underlying 
assumption  of  ABC  (Stevenson  and  Barnes, 
[20]).  With  accurate  information  about  the 
cost  of  activities,  these  costs  can  be  reduced. 
Additionally,  when  managers  begin  to  see  the 
costs  of  requested  products,  they  may  re¬ 
evaluate  the  need  for  the  products  and 
reduce  their  requests  thereby  making  costs 
decrease  for  the  same  level  of  output.  Cost 
knowledge  may  be  useful  in  discovering 
value  added  activities  from  non-value  added 
activities  as  well.  By  eliminating  non-value 
added  activities,  total  cost  may  decrease.  A 
decrease  in  total  costs  will  help  an 
organization  become  more  efficient  and 
allow  the  cost  savings  to  be  used  elsewhere. 

Financial  analysis  can  also  help  public 
entities  improve.  With  regular  records  of 
accounting  performance,  comparisons  such 
as  those  made  with  a  large  government 
bookstore  can  illustrate  areas  where 
improvements  can  be  made.  The  earlier 
example  points  to  asset  management  as  a 
potential  problem  area.  Comparing  its 
inventory  policies  to  those  of  its  private 
counterparts  may  reveal  some  ways  that  a 
large  government  owned  bookstore  could 
reduce  inventory  costs,  for  example. 

Having  accounting  records  also 
facilitates  rewarding  employees  for  good 
performance.  Table  1  shows  that  the  sales 
per  employee  at  a  large  government  owed 
bookstore  are  greater  than  the  sales  per 
employee  at  either  Barnes  and  Noble  or 
Crown  Books  ($220,000  versus  $100,000 
and  $180,000,  respectively).  This  is  an 
indication  that  the  USAFA  Bookstore  is 
probably  at  an  efficient  staffing  level,  and 
that  its  employees  are  productive.  Where 
management  needs  to  focus  its  attention  is 
on  asset  management. 


384 


If  efficiencies  can  be  generated, 
employees  should  be  rewarded.  Gainsharing 
is  a  program  used  to  allow  federal  employees 
to  share  in  savings  that  result  from  cost 
reductions.  This  program  allows  employees 
to  share  personally  in  a  percentage  of  the 
savings  they  generate.  When  efficiencies  can 
be  identified,  measured  costs  are  likely  to 
decrease.  The  gainsharing  program  within 
both  the  mission  and  support  sides  of  the 
USAF  may  see  cost  saving  ideas  increase 
and  pay-outs  for  these  ideas  increase. 

IMPLEMENT  A  TION  OF  INITIA  TIVES 

There  are  several  factors  influencing 
implementation  of  any  major  initiative. 
These  factors  include:  source,  clarity  and 
support  for  the  policy,  complexity  of  the 
administration,  incentives  for  the 
implementors,  and  resource  allocations. 
Policy  makers  and  policy  implementors  are 
normally  different  actors  with  different  goals 
and  motivations.  In  this  section,  we  briefly 
discuss  the  major  issue  associated  with 
implementation  of  the  previously  discussed 
initiatives. 

Privatization.  This  initiative  has  high 
level  support  and  its  currently  very  trendy. 
For  this  reason,  political  hurdles  are 
surprisingly  small.  Privatizing  remains 
complex,  however,  because  an  adequate 
basis  for  comparison  between  private  and 
public  organizations  is  extremely  difficult  to 
derive.  Financial  analysis  blunts  this  criticism 
but  does  not  remove  it.  The  allocation  of 
resources  becomes  an  issue  with  this 
alternative  because  a  public  good  is  being 
supplied  by  a  private  organization. 
Privatization  often  may  mean  that  certain 
government  facilities  get  less  use. 
Constituents  may  adversely  react  to  the 
closing  or  reduction  of  government  facilities 


feeling  that  their  local  economy  may  suffer 
from  lost  jobs  and  lower  tax  revenue.  Also, 
having  the  private  sector  deliver  a  public 
good  opens  up  a  debate  concerning  the  true 
function  of  government.  Lastly  and  perhaps 
most  significantly,  some  functions  shouldn't 
be  privatized  because  of  combat  capability 
concerns. 

Competition  Being  able  to 
successfully  implement  this  initiative  depends 
upon  having  an  accounting  system  in  place 
that  provides  the  visibility  into  costs  needed 
to  make  comparisons  among  various 
providers.  The  DBOF  system  has  political 
support  at  the  highest  levels  within  the 
USAF,  DOD,  and  Congress.  High  level 
support  makes  policy  implementation  easier. 
Provisions  for  DBOF  were  approved  in  the 
1991  Defense  Authorization  Act.  But,  the 
administrative  execution  of  that  1991  Act 
has  been  dismal  (GAO/AIMD-94-7R  [8]). 
There  is  no  central  organization  overseeing 
DBOF,  the  accounting  mechanisms  are  not 
in  place,  and  the  GAO  has  made  complaints 
about  cash  management  (GAO/AIMD-94-7R 
[8]).  Additionally,  the  DBOF  was  first 
implemented  with  two  goals  in  mind:  cost 
reductions  and  market-like  efficiency  (DOD 
Comptroller,  [4]).  To  date,  many  market¬ 
like  incentives  are  still  absent.  Legislatively 
established  prices  and  other  controls  are 
significant  political  hurdles  that  block  the 
way  to  efficiency.  The  issue  is  whether 
DBOF  will  provide  market-like  efficiency 
incentives  or  will  it  simply  be  used  as  a 
budget  reduction  tool.  There  is  support  at 
the  top  levels  for  DBOF,  but  support  for  it  at 
lower  organizational  levels  is  essentially  non¬ 
existent  (Seiden,  [19]).  If  market-like 
efficiency  is  a  true  goal,  then  actual 
implementation  will  become  much  more 
complex  as  the  DOD  gropes  with  adequate 
incentives  to  encourage  worker  support. 
Perhaps  top  managers  must  relinquish  some 
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direct  financial  control  to  push  decision¬ 
making  down  to  lower  levels.  For  example, 
commanders  could  be  given  the  power  to 
make  procurement  decisions  and  be  held 
accountable  for  financial  performance. 

On  one  hand,  implementing 
competition  should  not  harm  combat 
capability.  DOD  managers,  when  given  the 
ability  to  choose  suppliers,  will  theoretically 
choose  the  low  cost  option,  assuming  the 
quality  of  the  product  is  sufficient.  When 
government  suppliers  choose  transfer  prices 
that  are  too  high,  customers  will  have 
justification  to  find  other  private  suppliers. 
During  times  of  combat,  however,  some 
private  firms  may  choose  not  to  do  business 
with  the  USAF  due  to  the  risk  of  delivering 
goods  to  a  combat  zone  for  example.  From 
a  combat  capability  perspective,  keeping  an 
adequate  number  of  suppliers  who  can 
deliver  the  goods  and  services  needed  during 
times  of  conflict  is  a  crucial  concern.  This 
option  could  potentially  decrease  the 
availability  of  DOD  suppliers  and  hurt  US. 
combat  capability.  If  DOD  suppliers  cannot 
improve  efficiencies  to  remain  competitive, 
then  policy  makers  may  decide  to  subsidize 
them  in  order  to  have  sufficient  surge 
capability.  In  addition,  preferred  supplier 
lists,  the  shrinking  defense  industrial  base, 
bureaucratic  restrictions,  and  capital 
intensive  projects  all  limit  the  number  of 
potential  suppliers  of  goods  and  services  to 
the  USAF. 

Continuous  Improvement  Without 
motivation,  DOD  workers  have  little 
incentive  to  become  efficient.  In  many  cases, 
perverse  incentives  may  encourage 
inefficiency,  such  as  trying  to  secure  a  bigger 
budget  for  non-value  added  activity.  Part  of 
the  motivation  to  improve  may  come  from 
increased  competition  form  both  other 


government  providers  and  from  civilian 
contractors.  Part  of  the  incentive  may  come 
form  the  threat  of  having  their  function 
privatized.  Incentive  for  improvement  may 
also  be  derived  from  rewards  for 
improvements  in  performance. 

CONCLUSION 

In  this  study,  we  examine  three  policy 
initiatives  aimed  at  increasing  efficiency  in 
the  USAF.  We  also  introduce  financial 
analysis  as  a  means  to  compare  the 
operations  public  and  private  entities. 
Privatization,  competition,  and  continuous 
improvement  are  all  necessary  tools  as  the 
Air  Force  and  DOD  are  forced  to  become 
more  efficient  without  sacrificing  combat 
capability.  We  contend  that  management 
tools  such  as  ABC  and  financial  analysis  will 
play  a  growing  role  in  the  selection  of  these 
initiatives.  The  key  to  success,  however,  is 
in  having  the  adequate  cost  visibility  so  that 
comparisons  can  be  made  and  rewards  for 
improvement  can  be  given.  Thus,  the  efforts 
currently  under  way  to  reform  DOD 
accounting  procedures  must  continue. 

The  opinions  expressed  in  this  study  are 
those  of  the  authors  and  do  not  necessarily 
reflect  those  of  the  Department  of  Defense, 
the  Department  of  the  Air  Force,  the 
United  States  Air  Force  Academy  or  any 
other  government  agency. 

This  study  was  funded  in  part  by  a  grant 
from  the  Defense  Systems  Management 
College,  Ft  Belvoir,  VA. 
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TABLE  1 


This  table  shows  a  comparison  of  the  financial  ratios  of  the  Government  Owned  Bookstore  and 
the  two  private  firms,  Crown  Books  and  Barnes  and  Noble. 


Ratio  USAFA  Bookstore 

Crown  Books 

Barnes  and  Noble 

Asset  Turnover 
(sales/assets) 

0.3 

1.5 

1.5 

Inventory  Turnover 
(sales Anv) 

0.3 

4.0 

3.7 

Receivables  Turnover 
(sales/accts.  rec) 

0.9 

74.3 

64.7 

Sales  to  NWC 
(sales/net  working  cap) 

0.3 

2.6 

7.3 

Current  Ratio 
(curr  assets/curr  liab) 

16.5 

2.6 

1.5 

Debt  Ratio 
( tot  liab/tot  assets) 

0.06 

0.36 

0.63 

Sales  Per  Employee 
(in  thousands) 

$220 

$100 

$180 
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THE  FUTURE  OF  SOF  ACQUISITION:  IS  IT  A  GROWTH  INDUSTRY? 


Lieutenant  Colonel  Randolph  A.  Mathews 


INTRODUCTION 

"  The  clear  and  present  dangers  of  the  Cold  War  made  the  need  for  national  security  commitments 
and  expenditures  obvious...  today  the  task  of  mobilizing  support  for  national  security  priorities  has  become  more 
complicated." 

National  Security  Strategy,  1994 


Yogi  Berra,  the  venerable  American 
spokesman  and  prognosticator  stated, 
"the  future  ain't  what  it  used  to  be." 
Both  our  world  and  our  view  of  it  have 
dramatically  changed  in  the  last  five 
years.  The  threat  of  war  with  massive 
Soviet  forces,  or  a  nuclear  exchange 
across  the  poles  has  been  replaced  with 
a  complex  array  of  new  security 
challenges  unimaginable  only  ten  years 
ago.  While  our  relationships  with 
Russia  and  the  many  new  democracies 
of  the  former  Soviet  Union  are  good,  the 
successor  states  are  fraught  with 
economic  and  political  tensions  as  they 
come  of  age.  Transnational  threats  such 
as  narcotics,  refugee  flows,  terrorism 
and  the  proliferation  of  weapons  of 
mass  destruction  pose  significant 
problems  and  challenges  to  United 
States  security  policy  and  strategy.  As  a 
result,  we  Americans  have  had  to 
reassess  our  national  priorities;  foreign 
and  domestic  policies;  and,  as  a 
consequence,  how  we  allocate  our 
defense  resources.  As  the  dust  is  now 
settling  from  this  assessment  and 
reprioritization,  we  can  see  that  the 
scope  and  focus  of  defense  materiel 
acquisition  has  significantly  changed. 
This  paper  will  explore  this  change,  and 
the  efforts  within  the  Department  of 


Defense  (DoD)  and  the  Special 
Operations  Forces  (SOF)  to  deal  with 
the  new  realities  facing  defense. 

A  HISTORICAL  PERSPECTIVE 

"Good  defense  is  not  cheap  defense.” 

President  Dwight  D.  Eisenhower 

The  title  of  this  paper  questions  if  SOF 
acquisition  is  a  "growth  industry."  The 
question  is  more  complex  than  it 
initially  appears,  and  therefore  cannot 
be  properly  answered  in  a  single  word. 
It  is  more  constructive  to  seek  an  answer 
by  examining  where  it  was,  is,  and 
where  it  is  going.  First,  a  historical 
view:  from  1984  through  1994,  the 

research,  development  and  acquisition 
(RDA)  accounts  represented  an  average 
of  24%  of  the  total  Defense  Department 
budget  (Defense  Total  Obligation 
Authority).  Put  another  way,  the 
Department  of  Defense  invested 
between  19%  (FY  1994)  and  33%  (FY 
1984)  of  its  defense  budget  in 
modernization  efforts.  On  average,  this 
represented  a  split  of  13%  to  research 
and  development  activities  and  11%  to 
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acquisition  of  end  items.1  Our  national 
security  strategies  supported  this 
significant  modernization  effort,  which 
paid  great  dividends  in  the  dissolution 
of  the  Soviet  threat  and  the  military 
successes  in  Grenada,  Panama,  and  the 
Persian  Gulf.  Indeed,  the  successes  of 
the  Gulf  War  were  a  mixed  blessing  for 
modernization:  the  United  States  had 
an  internationally  televised  showcase 
for  its  technologically  superior 
weaponry,  giving  glowing  evidence  that 
technology  works,  but  the  fact  that  our 
current  technology  so  overpowered  the 
opposition  augured  against  further 
expensive  modernization  efforts. 

THE  CURRENT  DEFENSE 
ACQUISITION  SITUATION 

"Nozv,  with  the  DoD  procurement  budget 
reduced.. .it  is  critical  that. ..[we]  adopt  cost  reduction 
in  current  and  future  weapons  and  support  systems 
as  a  primary  objective. " 

Defense  Science  and  Technology  Strategy,  1994 

Looking  at  the  past  provides  a  basis  of 
comparison  for  today's  situation,  where 
RDA  amounts  to  18%  of  the  overall 
Defense  Department  budget,  of  which 
the  acquisition  portion  has  shrunk  to 
4%. 2  This,  however,  does  not  represent 
a  reckless  reduction.  Secretary  Perry 
has  methodically  sought  to  place  the 
inevitable  reduction  in  overall  defense 
spending  into  focus  with  his  five 
principal  priorities: 


1  All  figures  obtained  from  the  respective 
Department  of  Defense  Future  Years  Defense 
Plan. 

1  Ibid. 


•  Implement  the  bottom-up  force 
structure; 

•  Protect  the  force  to  sustain  a  strong 
readiness  capability; 

•  Redirect  the  modernization  program 
to  sustain  a  strong  science  and 
technology  program,  invest  in  next- 
generation  systems,  focus  on  near  to 
medium  requirements  for  theater 
missile  defense,  maintain  a  strong 
intelligence  program  and  maintain 
selected  elements  in  the  industrial  base; 

•  Execute  business  more  responsibly 
and  effectively;  and 

•  Reinvest  defense  dollars  in  dual-use 
technology.3 

The  first  two  items  demonstrate  DoD's 
resolve  to  preserve  an  adequate  and 
capable  force  structure.  In  the  area  of 
SOF,  there  has  been  no  significant 
change  to  active  force  structure  as  a 
result  of  the  bottom-up  review. 
Nevertheless,  the  Commander-in-Chief 
of  the  United  States  Special  Operations 
Command  (USSOCOM)  is  committed  to 
preserving  the  SOF  force  structure,  now 
considered  to  be  stretched  thin  by  the 
wide  variety  of  missions  facing  SOF  in 
today's  unstable  world.  The  implication 
of  preserving  force  structure  and 
readiness  is  that  resources,  which 
previously  could  be  identified  for 
modernization,  are  now  allocated 
against  other  priorities.  The  last  three 
items  address  the  future  emphasis  for 
defense  acquisition  and,  as  the  focus  of 
this  paper,  will  be  addressed  in  more 
detail  later. 


3  Testimony  of  Gary  L.  Denman,  Director, 
Advanced  Research  Projects  Agency  to  the 
House  Appropriations  Committee,  April  12, 
1994. 


392 


As  part  of  his  effort  to  implement 
Secretary  Perry's  priorities,  the  Deputy 
Secretary  of  Defense  asked  the  military 
departments,  in  August  1994,  to 
determine  possible  savings  from 
restructuring  or  cancelling  the  following 
programs: 

•  New  Attack  Submarine; 

•  DDG  51  Destroyers; 

•  Advanced  Amphibious  Assault 
Vehicle; 

•  V-22  Osprey  (in  which  SOF  is  a  key 
participant); 

•  RAH  66  Comanche  helicopter; 

•  Advanced  Field  Artillery  System; 

•  Tri-Service  Standoff  Attack  Missile; 

•  Joint  Primary  Aircraft  Training 
System;  and  the 

•  F-22  Fighter. 

He  has  stated  that,  "readiness, 
sustainability  and  quality  of  life  may 
require  us  to  shift  resources  from  some 
other  [budget]  priorities."4  This  clearly 
signals  a  further  reduction  in  our  major 
end  item  acquisition  efforts,  but  is  not 
inconsistent  with  Secretary  Perry's 
priority  to  redirect  modernization 
toward  science  and  technology 
programs.  In  real  terms,  the  federal, 
defense,  and  acquisition  budgets  are  all 
declining.  This  can  be  interpreted  as 
either  the  bellwether  of  a  continuing 
acquisition  decline,  or  the  basis  for  a 
future,  comprehensive  (albeit  smaller) 
modernization  program.  From  the 
viewpoint  of  defense  acquisition,  the 
latter  interpretation  appears  to  be 


4  Deputy  Secretary  of  Defense  Memorandum 
for  Members  of  the  Defense  Resources  Board, 
Subject:  Additional  DRB  Program  Alternatives, 
August  18, 1994 


supported  by  examining  ongoing  and 
planned  efforts  in  the  acquisition  arena. 

The  Deputy  Secretary's  request  has 
reinforced  the  fact  that  DoD  must  seek 
other  viable  avenues  by  which  it  can 
maintain  essential  elements  of  the 
industrial  base  and  preserve  readiness 
through  techniques  such  as  service  life 
extension  programs,  upgrades  to 
existing  systems  and  preplanned 
product  improvements  (P3I).  Although 
these  techniques  and  methods  have 
always  been  a  part  of  the 
comprehensive  defense  acquisition 
program,  they  were  previously  pursued 
less  often  than  new  procurement.  The 
military  departments,  however,  are  now 
anxious  to  preserve  their  technological 
edge  by  any  means,  and  therefore 
upgrades  currently  enjoy  a  renewed 
emphasis.  In  the  case  where  the  budget 
(and  especially  RDA)  dollars  must  be 
carefully  husbanded,  upgrading 
capable  equipment  rather  than 
replacing  it  with  new  equipment  may 
provide  the  most  "bang  for  the  buck." 

THE  CURRENT  SOF  ACQUISITION 
SITUATION 

"  The  shift  in  focus  from  the  Cold  War  and  an  overall 
decrease  in  the  size  of  the  defense  budget  has  led  to  a 
major  reevaluation  of  the  entire  USSOCOM 
modernization  program.  This  has  resulted  in  a 
much  leaner,  affordable  and  balanced  modernization 
program. " 

General  Wayne  A.  Downing,  Commander-in-Chief, 
USSOCOM 

Thus  far,  discussion  has  been  oriented 
toward  the  larger  Defense  budget.  In 
the  particular  case  of  the  Special 
Operations  Forces  (SOF),  it  is  instructive 
to  examine  the  relationship  between  its 
budget  and  RDA  accounts.  Since  its 
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establishment  as  a  separate  entity  eight 
years  ago,  the  SOF  budget  (Major  Force 
Program  11),  although  representing 
approximately  1%  of  the  overall  defense 
budget,  has  historically  had  few  of  the 
funding  decrements  which  have 
plagued  the  conventional  forces. 
However,  that  immunity  from 
reduction  has  ended.  For  the  1995-1999 
period,  SOF  RDA  was  reduced  $1.3 
billion,  or  33%  from  the  previously 
planned  levels.  The  future  is  no 
brighter.  A  glimpse  into  the  SOF 
program  shows  a  change  in  the  RDA 
share  from  40%  of  MFP  11  in  Fiscal  Year 
1992,  to  23%  in  Fiscal  Year  2001. 5  Since 
it  is  apparent  that,  as  with  the  overall 
DoD  budget,  SOF  acquisition  accounts 
are  shrinking,  specific  steps  must  be 
taken  to  keep  SOF  modernization  efforts 
viable. 

SOF  RESPONDS  TO  THE 
CHALLENGE 

" There  is  no  ideal  solution. ..one  must  select  that 
which  seems  best  from  the  most  varied  aspects  and 
then  pursue  it  resolutely.” 

Field  Marshal  Erwin  Rommel 

The  SOF  acquisition  community  has 
implemented  many  belt-tightening 
measures  to  maximize  the  effect  of  the 
now  scarce  investment  funds.  There  are 
traditional  measures,  such  as  stretching 
out  schedules  to  spread  the  investment, 
minimizing  the  impacts  of  reductions 
on  any  given  year,  or  cancelling  a 
program  to  use  the  funds  on  higher- 
priority  programs.  Neither  of  these 
approaches  are  as  meaningful,  however. 


5  Figures  taken  from  appropriate  years' 
Program  Objective  Memorandum  extracts. 


as  a  concerted  and  renewed  effort  to 
carefully  evaluate  the  true  value  and 
cost-effectiveness  of  a  materiel  solution 
to  a  requirement.  It  is  only  by 
judiciously  challenging  the 

requirements  and  improving  the 
acquisition  decision  processes  that  we 
can  make  the  most  of  our  RDA  dollars. 
In  the  past,  it  has  been  said  that  the  Joint 
Requirements  Oversight  Committee 
(JROC)  "never  saw  a  requirement  they 
didn't  like."  Times  have  changed  for  the 
JROC  and  for  the  SOF  community.  The 
Commander-in-Chief,  U.S.  Special 
Operations  Command  has  the  authority 
to  approve  any  SOF-peculiar  materiel 
requirements,  and  has  recently  refined 
the  requirements  review  and  approval 
process  to  focus  not  just  on  the 
individual  requirement,  but  its  impact 
across  and  among  all  systems  in  a  given 
mission  area.  If  the  capabilities  and 
requirements  of  an  entire  mission  area 
are  considered,  it  may  prove  that 
existing  systems  can  be  improved  or 
modified  to  fill  additional  requirements. 

Selective  upgrades  to  existing  systems 
may  be  more  cost-effective  than  new 
weapons  systems,  thus  freeing  up 
additional,  scarce  resources  for  other 
high-priority  items. 

The  foregoing  does  not  negate  the  fact 
that  modernizing  with  new  systems  is 
still  necessary,  and  often  the  most 
desirable  or  cost-effective  solution.  In 
the  broad  view,  SOF  require  mobility 
platforms;  command,  control, 
communications,  and  intelligence  (C3I) 
systems;  and  support  equipment.  The 
current  program  provides  for  new 
mobility  platforms  such  as  the  Mark  V 
Special  Operations  Craft,  the  Naval 
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Special  Warfare  Rigid  Inflatable  Boat, 
and  the  V-22  Osprey  aircraft.  New  C3I 
systems  include  the  Special  Operations 
Command  Research,  Analysis,  and 
Threat  Evaluation  System  (SOCRATES), 
the  SOF  Tactical  Assured  Connectivity 
System  (SOFTACS),  the  SOF 
Intelligence  Vehicle  (SOFIV),  and  the 
Multi-Mission  Advanced  Tactical 
Terminal  (MATT).  Support  equipment 
covers  a  wide  range  of  new  items  from 
the  SOF  offensive  handgun  to  the  SOF 
Planning  and  Rehearsal  System 
(SOFPARS).  Within  these  key 
acquisition  programs,  the  effect  of 
reductions  has  required  major 
restructuring.  However,  as  the  world 
security  situation  continues  to  change, 
the  materiel  requirements  likewise 
continue  to  evolve.  There  will  doubtless 
be  a  need  in  the  foreseeable  future  for 
some  level  of  new  systems  acquisition. 

As  the  Defense  Department,  and  by 
direct  extension  the  SOF,  is  charged 
with  an  ever-increasing  mission 
(witness  the  humanitarian, 

peacekeeping,  and  police  actions  in  the 
daily  news)  on  a  decreasing  budget, 
there  is  an  increased  emphasis  in  the 
Defense  Department  and  within  the 
SOF  community  in  the  area  of  selective 
upgrades,  service  life  extension 
programs,  P3I  and  horizontal 
technology  insertion.  These  approaches 
allow  selective  improvement  or 
maintenance  of  existing  systems,  and 
integration  of  emerging  or  more 
modern  technology  at  appropriate  and 
affordable  levels  to  keep  a  proven 
system  technologically  superior  to  the 
threat.  Within  SOF,  examples  of  this 
approach  include  radar,  infrared 


detector,  and  center  wing  box  upgrades 
for  our  aircraft;  and  the  Mod  1 
navigation,  propulsion,  and 

communication  upgrades  to  the  Mark  8 
SEAL  Delivery  Vehicle.  It  is  critical  to 
note  that  upgrading  proven  systems 
inserts  the  newest  technology  into  the 
force  structure  more  quickly  than 
developing  a  new  platform  or  system. 
In  the  case  of  technology,  the  challenge 
is  to  put  it  into  the  hands  of  the  SOF 
while  it  is  still  new,  allowing  our 
fighting  forces  to  keep  the  technological 
edge. 

INNOVATIVE  ACQUISITION 
METHODS 

"Shrinking  defense  budgets  dictate  that  we  can  no 
longer  afford  defense-funded,  defense-unicjue 
solutions  to  our  requirements.  We  must  utilize  the 
economies  of  scale  and  technology  innovation  of 
commercial  industry. " 

Defense  Science  and  Technology  Strategy,  1994 

The  defense  acquisition  system  has 
taken  a  good  deal  of  public  as  well  as 
internal  criticism  regarding  its 
unresponsiveness  and  inertia.  The 
Defense  Department  is  using  a  variety 
of  methods  within  the  defense 
acquisition  system  to  overcome  some  of 
the  perceived  shortfalls  and  expense  of 
the  traditional  process.  These 
approaches  include:  procuring  non- 
developmental  items 

(NDI)/commercial,  off-the-shelf  (COTS) 
items;  cooperative  programs;  leveraging 
other  programs;  and  design-to-cost 
contracts. 

In  the  name  of  "reinventing 
government,"  there  has  been  a  good 
deal  of  recent  publicity  about  the 
change  in  emphasis  from  military 
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specifications  and  design  specifications 
to  commercial  and  performance 
specifications.  In  fact,  the  use  of 
commercial  specifications  and 
commercial  products  has  been  a  part  of 
the  defense  acquisition  tool  kit  for  some 
time— in  the  form  of  NDI/COTS.  The 
term  NDI/COTS  is  a  generic  reference 
covering  materiel  available  from  a  wide 
variety  of  sources,  with  little  or  no 
development  effort  required  by  the 
government  In  the  SOF  arena,  there 
has  been  success  in  meeting  materiel 

requirements  with  commercial 

(militarized  or  ruggedized 

appropriately)  items,  adapted  or 

modified  military  systems,  and 
combinations  of  existing  commercial 
and  military  components.  This 
approach  is  a  favorable  acquisition 
strategy,  as  it  significantly  reduces 
developmental  costs,  often  reducing 
those  investments  to  simply  obtaining 
test  samples  or  prototypes  and  the 
required  test  and  evaluation  for 
suitability.  Examples  of  this  approach 
for  SOF  systems  include:  Mark  V  SOC, 
SOF  offensive  handgun,  underwater 
GPS,  and  intra-team  radios.  The  current 
interest  in  exploiting  dual-use 
technologies  also  adds  additional 
emphasis  on  COTS.  One  of  Secretary 
Perry's  priorities  is  to  reinvest  defense 
dollars  into  dual-use  technologies;  this 
refers  to  technologies,  processes,  and 
products  that  have  both  military  and 
non-military  applications.  The  goal  is  to 
achieve  militarily  superior  technology 
in  a  manner  which  benefits  both  classes 
of  products.  As  the  SOF  community 
capitalizes  on  this  thrust,  it  will  be 
better  prepared  to  defend  its  investment 
budget,  and  to  maintain  a  viable 
modernization  effort. 


Another  acquisition  strategy  which  has 
more  recently  come  into  favor  is  "design 
to  cost  (DTC)."  This  method  provides 
industry  with  a  requirement  couched  in 
terms  of  performance,  with  the  focus  on 
cost  and  performance  trade-offs  needed 
to  define  an  affordable  system  meeting 
or  exceeding  required  performance 
levels.  The  DTC  ideal  is  to  achieve  a 
proper  balance  between  development, 
production,  and  support  costs.  In 
today's  environment  of  constrained 
budgets,  SOF  will  rely  more  upon  this 
methodology  when  NDI/  COTS 

materiel  solutions  are  not  available. 
DTC  and  concerted  efforts  to  educate 
government  and  industry  program 
managers  that  cost  overruns  are  a  relic 
of  the  past,  with  fixed  or  restructured 
budgets  and  programs  the  rule  for  the 
foreseeable  future,  will  assist  in 
controlling  program  and  life-cycle  costs. 

SOF  acquisition  has,  and  will  continue 
to  use  other  service  programs  as  a 
vehicle  to  produce  derivative  SOF- 
peculiar  variants  of  common  materiel. 
This  method,  reducing  SOF  program 
overhead  and  R&D  costs,  leverages  the 
economic  buy  quantities  of  the  larger 
service  programs,  and  provides  some 
commonality  for  continuing  support. 
Additionally,  it  provides  industry  the 
economies  of  scale  and  a  base  over 
which  to  spread  their  R&D  and  up-front 
investments.  Current  examples  of 
leveraged  SOF  programs  are  the  V-22 
Osprey,  Quiet  Knight,  the  MH  60K,  and 
the  MH  47E  helicopters.  As  the  services 
continue  to  develop  systems  with 
potential  SOF  application,  we  should 
expect  more  of  these  leveraged 
programs. 
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Similar  to  the  leveraged  program 
concept  is  the  cooperative  program. 
This  is  yet  another  method  to  increase 
the  effect  of  our  RDA  investments  and 
leverage  developing  technology 
available  elsewhere.  In  a  cooperative 
program,  risk  and  cost  are  shared 
between  two  or  more  agencies  or 
governments.  For  example,  the  Office 
of  the  Assistant  Secretary  for  Special 
Operations  and  Low-Intensity  Conflict 
executes  a  counter-terror  technology 
development  program.  The  program 
has  twenty  government  agencies  and 
three  foreign  governments 

participating,  which  benefit  by  the 
shared  costs  as  well  as  the  end  product 
technology. 

SCIENCE  AND  TECHNOLOGY  BASE 

"We  are  not  the  only  nation  with  competence  in 
defense  science  and  technology.  To  sustain  the  lead 
which  brought  us  victory  during  DESERT  STORM, 
...we  must  invest  in  the  next  generation  of  defense 
technologies. " 

William  J.  Perry,  Secretary  of  Defense 

In  addition  to  the  classical,  end-item- 
oriented  acquisition  programs,  it  is 
essential  to  maintain  a  viable  science 
and  technology  base  from  which  to 
exploit  emerging  and  promising 
technologies.  Two  programs  which  are 
useful  examples  of  DoD's  efforts  to 
nurture  technology  are  the  Independent 
Research  and  Development  and  the 
Advanced  Concept  Technology 
Demonstration  programs. 

Independent  Research  and 

Development  (IR&D)  is  a  program 
which  provides  an  incentive  for 
industry  to  pursue  research  at  its  own 
expense  and  risk,  with  the  hope  that  a 
portion  of  the  research  cost  will  be 


recovered  against  current  or  future 
contracts.  A  1988  Defense  Science 
Board  study  reported  that  "there  is 
probably  no  other  mechanism  that  is 
more  effective  in  developing  and 
inserting  technology  into  defense 
systems  than  IR&D."6  Congress  inserted 
language  into  the  1992  Appropriation 
bill  to  raise  the  split  for  government 
payment  from  80  to  100  per  cent  by 
1995,  which  should  provide  additional 
impetus  for  bringing  new  technology 
into  SOF  during  difficult  budgetary 
times. 

Yet  another  method  of  previewing 
promising  technologies  "on  the  cheap"  is 
the  Advanced  Concept  Technology 
Demonstration  (ACTD).  These 

"fieldable  prototypes"  are  a  precursor  to 
an  acquisition  program,  which  can 
assist  in  refining  user  and  developer 
understanding  of  new  concepts  and 
capabilities,  and  provide  a  smoother 
transition  from  development  to 
operational  utility.  When  ACTDs  are 
executed  with  militarily  significant 
quantities,  forces  are  better  prepared  to 
rapidly  incorporate  new  systems.  The 
demonstrations  provide  savings  in  two 
ways:  first,  detailed  requirements  are 
established  before  invoking  the  formal 
acquisition  process  (or  formal  program 
documentation),  and  secondly,  a  sound 
basis  is  provided  for  tailoring  entry  into 
the  acquisition  process.  ACTD  are  an 
especially  important  path  for 
introducing  new  technology  when  small 
quantities  are  required,  as  is  often  the 
case  with  SOF. 


6  The  Role  of  Demonstration  Approaches  in 
Acquisition  Reform,  Larry  Lynn,  Acquisition 
Review  Quarterly,  Spring  1994. 
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Finally,  there  is  Secretary  Perry's 
initiative,  the  Revolution  in  Military 
Affairs  (RMA),  in  which  SOF  is 
represented  by  the  Close  Battle  and 
Low-Intensity  Conflict  task  forces.  This 
initiative  is  intended  to  provide  an 
"over-the-horizon"  view  of  the  world 
situation  and  military  requirements  for 
the  mid-future  (2020).  The  RMA  is  seen 
as  a  way  in  which  the  DoD  can  realize 
dramatic  improvements  in  military 
effectiveness  and  combat  potential.  It 
encompasses  aggressive  development 
and  incorporation  of  highly  advanced 
technology  (previously  referred  to  as 
the  Military  Technical  Revolution), 
doctrinal  innovation  and  organizational 
adaptation  to  employ  these  emerging 
technologies  and  opportunities.  It  will 
help  DoD  and  SOF  focus  efforts  on  the 
most  promising  combinations  of 
technologies  and  doctrine  for  future 
forces.  While  primarily  a  intended  as  a 
defense  and  technology  strategic  vision, 
it  does  recognize  a  need  to  continue 
developing  militarily  significant 
technologies. 


CONCLUSION 

"My  pledge.. .is  to  have  the  best  trained,  best 
equipped  and  most  effective  forces  in  the  world.  But 
to  do  this,  with  reduced  resources,  we  have  to 
manage  it  right.  This  time  we  have  to  manage  it 
right.” 

William  J.  Perry,  Secretary  of  Defense 

The  future  of  SOF  acquisition  will  bring 
with  it  several  inevitable  facts.  First, 
there  will  be  fewer  new  start  programs. 

Upgrades,  modifications  to  existing 
platforms,  SLEP  and  P3I  will  replace 
new  systems  as  the  first  choice  for 


satisfying  new  requirements.  There  will 
still  be  new  start  programs,  but  they 
will  be  the  exception,  rather  than  the 
rule.  This  reduced  level  of  activity  is 
already  reflected  by  a  second  fact:  there 
will  be  fewer  defense  contractors.  The 
headlines  have  reinforced  this  fact- 
witness  the  recent  mergers  of  major 
defense  contractors.  Matters  of  market 
share,  efficiency,  and  economy  have 
made  this  consolidation  and 
downsizing  inevitable,  but  not  entirely 
unhealthy.  The  next  fact,  that  there  will 
be  less  funding  for  each  program  can  be 
seen  in  some  of  the  acquisition 
strategies  and  methods  addressed 
earlier.  Contracting  instruments  and 
contract  types  will  reflect  the  need  to 
maximize  dollar  value  and  eliminate 
"no-value-added"  activity.  Finally,  the 
capstone:  reduced  resources  will  require 
better  decision  processes.  Both  DoD 
and  SOF  recognize  the  need  to  make 
careful,  but  innovative  acquisition 
decisions.  These  decisions  must 
expedite  the  process  without  sacrificing 
the  safeguards  which  ensure  that  the 
department  fields  the  finest  possible 
materiel  to  the  troops.  The  fiscal 
realities  and  the  lessons  of  total  quality 
management  point  to  streamlined 
processes,  acceptance  of  program  risk, 
and  an  intense  scrutiny  of  each  decision. 

While  generally  the  forecast  for  future 
Defense  acquisition  is  not  as  bright  as  in 
previous  years,  the  SOF  community  is 
seeking  out  every  opportunity  and 
venue  to  keep  the  SOF  modernized  and 
capable.  As  new  roles  for  SOF 
materialize  daily,  so  do  opportunities 
for  new  technology  application  and 
force  enhancement.  The  challenges  are 
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to  carefully  consider  each  and  every 
requirement,  each  and  every 
technological  innovation  or  application; 
determine  how,  within  the  constrained 
resource  environment,  they  fit  relative 
to  other  opportunities  and 
requirements;  and  lastly  satisfy  them  by 
the  most  efficient  or  cost-effective 
means  possible. 

As  has  been  outlined,  SOF  acquisition  is 
still  in  business.  There  are  ongoing  and 
planned  acquisition  programs  well  into 
the  next  decade.  While  SOF  will 
continue  to  "do  business,"  it  must 
sharpen  its  focus,  embrace  the 
principles  espoused  by  the  Secretary  of 


Defense,  and  make  the  difficult 
decisions  required  to  comply  with  fiscal 
reality.  If  SOF  is  successful  in 
separating  the  important  "needs"  from 
the  many  "wants,"  and  applies  some  of 
the  methods  previously  outlined,  it  will 
be  able  to  continue  the  SOF 
modernization  program.  As  Babe  Ruth 
noted,  "yesterday's  home  runs  won't 
win  tomorrrow's  ball  game."  Given 
careful  management  and  innovative 
approaches,  one  concludes  that  while 
neither  defense  acquisition  at  large,  nor 
SOF  acquisition  in  particular  may  be 
considered  a  "growth  industry,"  SOF 
acquisition  will  remain  viable  well  into 
the  twenty-first  century  and  does, 
indeed,  have  a  future. 
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PUBLIC  LAW  101-510:  ITS  IMPACT  ON  THE 
DEFENSE  CONTRACTING  COMMUNITY 

Thomas  Meagher  &  James  Setlock 
TASC 


ABSTRACT 

The  past  two  years  have  seen  a 
resurgence  in  the  performance  of  aerospace 
and  defense  companies.  Profits  and  cash 
flow  are  up  dramatically,  resulting  in  higher 
stock  prices.  This  is  due  largely  to  cutbacks 
in  head  count  as  well  as  reductions  in  capital 
investment  and  R&D  spending.  As  we  move 
into  the  second  half  of  the  decade  however, 
conditions  do  not  appear  optimal  for 
continued  profit  growth.  Although  work 
force  attrition  will  continue,  the  largest  cuts 
are  behind  the  industry  while  capital 
equipment  and  R&D  spending  could 
decrease  even  further,  thus  impacting  future 
growth.  Given  the  inevitable  pressure  on 
revenues  resulting  from  defense  budget  cuts, 
it  appears  likely  that  operating  and  financial 
performance  will  be  negatively  impacted, 
accelerating  the  trend  towards  industry 
consolidation.  However,  there  is  another 
reason  why  industry  profitability  and  cash 
flow  might  be  significantly  impacted  relating 
to  changes  in  legislation  that  we  believe  may 
not  be  fully  understood  by  defense 
contractors. 

INTRODUCTION 

The  Demise  of  the  M-Accounts 

Prior  to  FY  90,  appropriations  that  had 
lapsed,  thus  reaching  the  end  of  their  legal 
availability,  were  then  placed  in  what  was 
known  as  the  Merged  or  M-accounts.  The 
funds  in  these  M-accounts  could  then  be 


used  to  finance  other  projects,  such  as 
contract  closeouts  and  equitable  adjustments. 
Many  observers,  particularly  those  on 
Capitol  Hill,  believed  that  M-account 
monies  were  being  redirected  to  fund  other 
programs  that  had  not  received  the  desired 
levels  of  resources  through  the  authorization 
and  appropriation  process.  Therefore, 
beginning  with  the  FY  90  defense 
authorization  bill,  M-accounts  were 
eliminated.  The  effect  on  acquisition 
programs  since  that  time  has  been  two  fold. 
Funds  originally  obligated  and  not  disbursed 
have  now  been  “cancelled”,  thereby 
eliminated  from  being  used  to  pay  for 
outstanding  deliverables. 

In  addition,  if  government  personnel 
determine  that  products  and  services 
previously  billed  and  paid  for  were  disbursed 
from  cancelled  appropriations,  contractors 
could  be  required  to  refund  those  monies  on 
a  temporary  basis.  In  both  cases, 
government  contracting  officers  are  being 
forced  to  ask  for  additional  current  year 
funding  to  pay  for  requirements  which  were 
believed  to  have  been  fully  funded. 

The  severity  of  the  problem  cannot  be 
overestimated.  Based  on  experience,  we 
believe  that  since  fiscal  year  1991  hundreds 
of  millions  of  dollars  in  previous  year 
obligations  and  disbursements  may  be  at  risk. 
To  the  casual  observer  these  numbers,  when 
compared  to  the  defense  budget  as  a  whole, 
may  seem  inconsequential.  But  when  one 
narrows  the  comparison  to  just  the  R&D 
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and  procurement  budgets,  the  impact 
becomes  much  greater. 

Given  projected  spending  levels  over  the 
next  few  years,  DOD  could  find  itself 
exceeding  the  1%  restriction  on  the  use  of 
current  year  funds  to  pay  cancelled  year  bills 
(per  Public  Law  101-510).  However, 
legislative  relief  may  be  necessary  if  this 
ceiling  is  exceeded. 

The  financial  consequences  for  contractors 
could  be  equally  burdensome.  With  revenues 
decreasing  and  a  significant  part  of  the 
industry  restructuring  already  complete, 
contractors  have  been  looking  to  preserve 
cash  flow  as  a  way  to  fund  defense  related 
acquisitions  and  pursue  non-defense  oriented 
diversification.  Any  disruption  in  cash  flow 
generation  could  severely  hamper  these 
efforts.  Contract  closeouts  likewise  could  be 
delayed  or  invalidated  if  deliverables  and/or 
services  are  determined  to  have  been  paid 
incorrectly.  The  delays  of  contract  closeouts 
on  supporting  subcontractors  will  have  an 
addition  ripple  effect  on  prime  contractors. 
Prime  contractors  cannot  close  out  their 
government  contracts  until  all  of  their 
respective  subcontracts  have  been  closed 
out.  Administrative  costs  associated  with 
keeping  contracts  and  subcontracts  open  as 
well  as  delaying  final  payment  of  contracts 
for  subcontractors  have  a  significant  negative 
impact  on  cash  flow.  Even  after  all  contract 
deliverables  have  been  shipped,  contracts  can 
take  in  excess  of  ten  years  before  receiving 
final  close  out  approval. 

The  Impact  of  the  1989  Defense 
Management  Review 

The  ongoing  streamlining  of  Defense 
Logistic  Agency  (DLA)  payment  functions 
has  contributed  to  the  confusion.  As  a  result 
of  the  1989  Defense  Management  Review 


conducted  by  then  Secretary  of  Defense 
Cheney,  DLA  established  a  finance  center  in 
Columbus,  Ohio  to  consolidate  contract 
payment  responsibilities  previously  handled 
by  20  regional  sites.  Contract  administration 
continues  to  be  handled  by  five  regional 
Defense  Contract  Management  Districts 
(DCMD).  A  March  1994  GAO  report 
(GAO/NSIAD-94- 106)  documented  the 
difficulty  DLA  has  had  in  implementing  this 
consolidation. 

At  the  request  of  Sen.  John  Glenn  (D-OH), 
Chairman  of  the  Senate  Committee  on 
Governmental  Affairs,  the  GAO  researched 
some  $392M  out  of  a  total  of  $75 1M  in 
returned  checks  from  contractors  during  a 
six  month  period  ending  in  April  of  1993. 
The  report  noted  that  of  the  $392M,  some 
$305M  were  over  payments  related  to 
invoices  being  paid  without  recovering 
progress  payments  (58%  of  total),  duplicate 
payments  (12%)  and  other  errors  (30%). 
The  report  goes  on  to  indicate  that  virtually 
all  of  the  overpayments  were  discovered  by 
contractors. 

DLA  has  strengthened  its  reconciliation  team 
at  the  Defense  Finance  &  Accounting  Center 
(DFAS),  in  Columbus,  Ohio  to  deal  with  this 
problem.  In  all  fairness  to  DFAS,  however, 
the  GAO  report  further  noted  that  the  Center 
has  been  severely  hampered  by  a  huge 
backlog  of  problem  contracts  (estimated  at 
6,600  in  total),  the  departure  of  experienced 
employees  (necessitating  the  hiring  of  new 
personnel  who  require  time  to  get  up  to 
speed)  and  the  accelerated  transfer  of 
contracts  to  the  Mechanization  of  Contract 
Administration  (MOCAS)  management 
system  from  other  DFAS  locations,  which 
overwhelmed  the  ability  of  Center  personnel 
to  manage  these  contracts. 
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The  Impact  on  Current  DFAS  Policies 
and  Procedures 

In  particular,  we  believe  problems  associated 
with  the  transfer  from  other  systems  such  as 
the  Acquisition  Management  Information 
System  (AMIS)  to  MOCAS  to  be  of  critical 
importance.  AMIS  is  used  to  monitor 
contracts  out  of  Wright  Patterson  AFB  and 
other  Air  Force  facilities.  In  order  to  more 
efficiently  use  AMIS  data,  a  special 
application  known  as  the  Contractor 
Obligation  and  Liquidation  Tracking  System 
(COLTS)  has  been  developed.  COLTS  data 
output  offers  summary  analysis  regarding  the 
status  of  obligations  and  disbursements.  The 
data  identifies  the  potential  problems 
associated  with  unliquidated  obligations 
(ULOs),  under/over  payments  and  contract 
closeouts. 

Designed  by  TASC  (The  Analytic  Sciences 
Corporation)  for  the  Air  Force  Materiel 
Command  /  Aeronautical  Systems  Center 
(AFMC/ASC),  COLTS  has  the  ability  to 
map  out  the  relationship  between  the 
relevant  account  appropriations,  contract 
modifications,  accounting  classification 
reference  numbers  (ACRNs),  contract  line 
item  numbers  (CLINs)  and  shipment  / 
invoice  numbers,  thus  providing  the  user 
with  a  desk  top  tool  for  resolving 
contractual/financial  disconnects. 

Derivation  of  the  correct  ACRN  /  CLIN 
relationship  per  the  contract  is  critical  in 
determining  what  products  and  services  have 
been  paid  correctly  in  addition  to  being  able 
to  forecast  the  impact  of  appropriation 
cancellations  on  the  funding  of  future 
deliverables.  When  MOCAS  does  not 
provide  an  ACRN/CLIN  relationship,  these 
relationships  must  be  derived  manually.  The 
process  used  to  develop  the  ACRN/CLIN 
relationship  requires  that  the  contract  and  all 


modifications  be  available  and  that  the 
individual  developing  the  ACRN/CLIN 
relationships  review  in  sequential  date  order 
each  obligating  document.  Some  contracts 
have  an  excess  of  1,000  modifications  and  it 
will  take  a  substantial  amount  of  time  to 
manually  develop  these  respective 
ACRN/CLIN  relationships.  The  problem  is 
expected  to  worsen  as  new  contracts  transfer 
in  from  other  payment  systems, 
supplementing  the  400,000  plus  contracts 
currently  active  in  the  MOCAS  system. 

As  the  GAO  noted,  contractors  have 
assumed  the  bulk  of  responsibility  for 
identifying  incorrect  payments.  One  company 
reportedly  resorted  to  setting  up  a  special 
account  in  which  to  hold  known 
overpayments  in  lieu  of  expected  demand 
letters;  at  one  time,  the  balance  of  this 
account  was  approximately  S1.9M.  But  what 
happens  when  contractors  receive  a  demand 
letter  relating  to  overpayments  they  were  not 
aware  of? 

The  contractor  receives  a  summary  notice 
and  limited  documentation  stating  the 
amount  of  the  requested  refund  to  be 
returned  within  thirty  days  to  avoid  the 
imposition  of  interest  charges.  Yet,  the 
contractor  has  no  access  to  the  MOCAS 
system  to  understand  how  the  justification 
for  the  demand  amount  was  derived.  This 
information  however  is  available  through  the 
government  contracting  officer  and  identifies 
contract  obligations,  disbursements,  progress 
payments,  recoupments  and  internal 
balancing  adjustments. 

The  major  supporting  documentation  within 
the  MOCAS  system  consists  of  the 
Contingent  Liability  Register  (CLR), 
Contract  History  and  Contract  Abstract.  The 
CLR  presents  a  contract  summary  of  the 
ACRN  level  of  obligations  and  unliquidated 
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obligation  balances  (the  difference  between 
obligations  and  disbursements  by  ACRN). 
The  Contract  History  provides  a  transaction 
level  summary  of  obligation  and 
disbursement  activity  for  a  contract.  The 
balances  from  these  transactions  are  then 
reflected  in  the  CLR.  The  Contract  Abstract 
is  a  summary  of  activity  for  a  contract  and  is 
the  source  of  information  about  ACRNs, 
CLINs,  shipments  /  invoices  and  other 
administrative  data.  Receipt  of  this 
information  and  other  supporting 
documentation  from  the  contracting  officer 
will  assist  the  contractor  in: 

•  identifying  inconsistencies  between  the 
contract  documentation  and  what  was 
posted  in  DoD  financial  records 

•  correcting  mathematical  errors  in 
obligations,  disbursements,  progress 
payments  and  associated  recoupments 

•  determining  justification  for  internal 
balancing  adjustments. 


CONCLUSION 

With  the  elimination  of  the  M-accounts,  the 
accelerated  transfer  of  contracts  into  DFAS- 
Columbus,  the  departure  of  experienced 
personnel  and  the  planned  consolidation  of 
different  military  department  payment 
systems  into  one  centrally  managed  by  the 
DFAS,  payment  problems  will  likely  become 
worse.  It  will  be  incumbent  upon  defense 
contractors  and  contracting  officers  to 
devote  sufficient  resources  to  manage  the 
challenges  that  will  inevitable  arise  from 
having  to  do  more  with  less. 
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Measuring  Operational  Contracting  Cost,  Output,  and 

Quality  Together 

Karen  D.  Sorber,  CPCM,  Logistics  Management  Institute 
Dr.  Ronald  L.  Straight,  Howard  University 


ABSTRACT 

Performance  unit  costing  (PUC)  considers  the 
cost  of  operations  relative  to  performance  units. 
In  procurement,  performance  units  represent 
completed  actions,  adjusted  for  quality  by  an 
index  measuring  such  aspects  as  customer  satis¬ 
faction,  increased  use  of  commercial  products, 
or  attaining  goals  in  competition  or  socioeco¬ 
nomic  programs.  In  this  way,  cost,  output,  and 
quality  considerations  are  traded  off.  Applying 
PUC  represents  a  significant  shift  away  from 
simply  measuring  a  single  productivity  factor 
(such  as  procurement  administrative  lead-time) 
or  from  using  quantity  or  dollar  volume  as  a 
workload  indicator.  The  Logistics  Management 
Institute  (LMI)  uses  PUC  to  integrate  the  meas¬ 
urement  of  outputs  as  opposed  to  inputs,  quality 
as  determined  by  the  customer,  and  the  cost  of 
operations  —  a  crucial  process,  especially  in  a 
time  of  budget  reductions. 

Before  implementing  any  measurement  system, 
it  is  important  to  perform  some  sensitivity 
analysis  to  determine  the  true  incentives  gener¬ 
ated  by  that  system.  Especially  when  combin¬ 
ing  measures,  managers  must  take  great  care 
that  the  ranges  of  performance  and  weights 
applied  to  each  measure  lead  to  desired  out¬ 
comes.  Furthermore,  contract  specialists  must 
understand  the  effects  of  actions  that  they  take 
in  order  for  a  system  to  have  the  power  to  moti¬ 
vate  improved  performance. 

The  conclusions  in  this  paper  are  important  to 
contract  managers  and  others  concerned  with 
resource  allocation.  Regardless  of  the  perform¬ 
ance  evaluation  system  chosen,  managers  need 
to  come  to  terms  with  the  measurement  and 
motivational  issues  raised  in  this  paper. 


INTRODUCTION 

Measurement  systems  have  long  been  a  staple 
management  tool  for  business,  with  bottom-line 
profit  often  being  considered  the  primary  indi¬ 
cator  of  performance.  If  contracting  offices 
worked  in  a  competitive  environment,  offering 
services  to  those  customers  willing  to  pay  for 
them  on  a  fee-for-service  basis,  a  bottom-line 
evaluation  might  be  suitable.  [1,2]  However, 
for  monopoly  operations,  such  as  the  provision 
of  government  services  in  a  not-for-profit  envi¬ 
ronment,  the  bottom  line  may  not  be  sufficient 
or  even  calculable.  Regardless  of  this  situa¬ 
tion —  or  perhaps  because  of  the  need  to  be 
more  accountable  to  the  taxpayer  for  results  — 
demand  for  better  measurement  systems  for 
government  operations  has  increased.  The  idea 
that  government  should  run  like  a  business  — 
or  at  least  adopt  businesslike  principles  —  is  not 
new,  but  it  has  recently  gained  momentum.  In 
addition  to  the  Presidential  and  congressional 
compaigns  of  the  1990s  that  embraced  these 
concepts,  they  are  supported  by  many  other 
specific  actions.  For  instance, 

•  Congress  passed  the  Chief  Financial  Offi¬ 
cers  (CFO)  Act  in  1990,  which  requires  14 
of  the  largest  agencies  to  develop  meas¬ 
urement  systems  that  strengthen  system 
accounting;  enhance  internal  audits;  in¬ 
corporate  future  costs  of  decisions;  im¬ 
prove  management  of  cash,  credit,  inven¬ 
tory,  and  debt  collection;  and  charge 
customers  true  costs;  [3] 

•  the  Department  of  Defense  (DoD)  Comp¬ 
troller  [4]  and  the  Office  of  Management 
and  Budget  (OMB)  [5]  have  issued  guid¬ 
ance  to  carry  out  the  CFO  Act; 
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•  Vice  President  Gore’s  Report  of  the  Na¬ 
tional  Performance  Review,  1993,  [6] 
calls  for  creating  a  government  that  meas¬ 
ures  performance,  puts  customers  first, 
creates  competition  within  government,  is 
more  market-oriented,  and  is  decentral¬ 
ized; 

•  Congress  passed  the  Government  Per¬ 
formance  and  Results  Act  of  1993,  [7] 
which  emphasizes  the  need  for  perform¬ 
ance  measures  tied  to  organizational  stra¬ 
tegic  plans  and  objectives; 

•  the  Governmental  Accounting  Standards 
Board  [8]  has  developed  a  uniform  ap¬ 
proach  to  reporting  service  efforts  and  ac¬ 
complishments  that  considers  cost  per  unit 
of  outcome; 

•  the  National  Academy  of  Public  Admini¬ 
stration  [9]  and  the  American  Society  for 
Public  Administration  [10]  have  passed 
resolutions  supporting  additional  perform¬ 
ance  measurement  efforts  that  monitor 
program  quality  and  outcomes  as  part  of 
an  overall  system  aimed  at  improving  the 
performance  and  credibility  of  major  pub¬ 
lic  programs; 

•  Executive  Order  12862,  “Setting  Cus¬ 

tomer  Service  Standards,”  of  September 
11,  1993,  [11]  requires  (among  other 

things)  identifying  customers,  surveying 
them  to  determine  the  kind  and  quality  of 
services  they  want  and  their  level  of  satis¬ 
faction  with  existing  services,  posting 
service  standards  and  measuring  results 
against  them,  benchmarking  customer 
service  performance  against  the  best  in 
business,  and  providing  customers  with 
choices  in  both  the  sources  of  service  and 
the  means  of  delivery; 

•  Executive  Order  12931,  “Federal  Pro¬ 

curement  Reform,”  of  October  13,  1994, 
[12]  requires,  among  other  things,  a  focus 
on  measurable  results,  increased  attention 


to  understanding  and  meeting  customer 
needs,  and  an  increase  in  the  use  of  com¬ 
mercially  available  items;  and  finally, 

•  several  executive  agencies  have  a  number 
of  initiatives  under  way  to  comply  with 
the  statutes  that  have  been  cited  and  to 
improve  agency  operations,  such  as  the 
procurement  performance  measurement 
model  of  the  Procurement  Measurement 
Action  Team,  discussed  below.  [13] 

Since  quality  and  customer  service  are  inextri¬ 
cably  related  aspects  of  any  measurement  sys¬ 
tem,  failing  to  evaluate  them  with  efficiency  or 
production  may  result  in  suboptimized  overall 
performance.  Classically,  efforts  to  apply  busi¬ 
ness  management  to  government  operations 
have  resulted  in  pressing  the  government  man¬ 
ager  to  provide  the  current  level  of  service  with 
a  reduced  budget.  Now  the  goal  is  even  more 
difficult  to  attain:  providing  increased  service 
while  still  reducing  the  cost.  But  when  we  enter 
this  arena,  we  face  difficult  problems.  It  is  im¬ 
portant  to  consider  three  points  regarding  serv¬ 
ice  quality  as  viewed  by  the  customer: 

•  The  only  criteria  that  count  in  evaluating 
service  quality  are  defined  by  customers. 
Specifically,  service  quality  perceptions 
stem  from  how  well  a  provider  performs 
vis-a-vis  customers’  expectations  about 
how  well  the  provider  should  perform. 

•  Service  quality  is  much  more  difficult  for 
customers  to  evaluate  than  product  qual¬ 
ity,  because  of  its  inherently  subjective 
nature. 

•  Customers  do  not  evaluate  service  quality 
solely  on  the  basis  of  the  outcome  of  a 
service  rendered;  they  also  consider  the 
process  of  service  delivery  (for  instance, 
how  involved,  responsive,  and  friendly  the 
service  provider  is).  [14] 

Because  measuring  customer-driven  quality 
requires  comparing  customers’  expectations  to 
actual  performance,  government  measurement 


406 


systems  designed  without  a  customer  service 
dimension,  adequate  in  the  1980s,  are  no  longer 
so.  In  fact,  the  customers  or  stakeholders  of  the 
contracting  process  may  be  several  disparate 
groups  of  people:  broadly  defined,  customers 
are  not  only  the  project  officers  and  program 
managers,  but  also  contract  managers,  upper 
management,  the  agency  head,  Congress,  and 
the  public  at  large.  (For  our  purposes,  the  proj¬ 
ect  officer  probably  serves  as  the  best  example 
of  a  primary  customer.  It  should  be  assumed 
that  the  project  officer  is  carrying  out  the  goals 
of  the  program  as  directed  by  the  Congress, 
which  represents  the  general  public,  and  the  ex¬ 
ecutive  agency.) 

Since  resources  (usually  tax  revenues)  are  not 
available  to  meet  customers’  expectations  be¬ 
yond  a  certain  level,  managers  need  also  to 
measure  the  efficiency  with  which  they  are 
providing  services.  Contract  managers  have 
often  performed  this  function  by  measuring  the 
number  of  contract  awards  made  by  each  con¬ 
tract  specialist  or  how  quickly  contracts  are 
awarded.  When  these  measures  are  used  in 
isolation,  however,  the  effect  may  be  an  unfor¬ 
tunate  one  —  the  hurried  production  of  a  high 
number  of  poor-quality  contracts  that  require 
expending  an  extensive  effort  (often  by  a  differ¬ 
ent  group  of  people)  to  correct  their  inadequa¬ 
cies.  This  can  truly  suboptimize  overall  per¬ 
formance  and  waste  valuable  resources. 

To  avoid  the  use  of  measurement  systems  that 
could  lead  to  these  counterproductive  results, 
the  performance  unit  costing  (PUC)  approach 
combines  a  number  of  measures.  [15]  This 
concept  is  being  applied  in  fields  other  than 
procurement  by  the 

•  Air  Mobility  Command,  which  uses  it  to 
consolidate  various  measures  such  as 
hauling  cargo,  moving  passengers,  and 
providing  pilot  training; 


•  Military  Traffic  Management  Command, 
which  uses  it  to  measure  the  quality  of 
transportation  of  goods; 

•  National  Aeronautical  and  Space  Ad¬ 
ministration  (Ames  Research  Center), 
which  uses  it  to  combine  cost,  output,  and 
quality  as  a  total  performance  measure¬ 
ment  framework;  and 

•  Department  of  Energy  Southwest  Power 
Administration,  which  uses  it  for  perform¬ 
ance  budgeting. 

This  paper  starts  with  a  discussion  of  historical 
contracting  measures,  next  describes  the  PUC 
model  development  in  a  contracting  environ¬ 
ment,  and  then  amplifies  it  to  show  its  sensitiv¬ 
ity  to  the  measures  and  weights  selected  by 
managers. 

THE  MEASUREMENT  BASELINE 

Over  the  years,  others  have  proposed  many 
separate  measures  for  application  to  the  pro¬ 
curement  area. 

Commercial  Purchasing  Benchmarks 

The  Center  for  Advanced  Purchasing  Studies  of 
the  National  Association  of  Purchasing  Man¬ 
agement  produces  benchmarks  to  be  applied  to 
purchasing  in  various  industries.  [16]  A  group 
of  purchasing  professionals  from  each  industry 
develops  a  list  of  common  industry  bench¬ 
marks.  They  collect  and  report  data  on  the 
common  benchmarks  and  compile  summary 
statistics  for  association  members  and  academe. 
Some  measures  applicable  to  both  the  public 
and  private  sectors  are  shown  in  Exhibit  1 . 

Malcolm  Baldrige  Award  Criteria 

The  procurement  function  in  an  organization, 
like  the  rest  of  the  organization,  needs  to  have  a 
quality-management  system.  One  well-accepted 
way  of  defining  and  evaluating  a  quality 
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Exhibit  1. 

Summary  of  Commercial  Purchasing  Benchmarks 


Benchmark 

Median 

Mean 

Range 

Number 

1 .  Total  purchasing  (the  dollars  spent  with  suppliers)  accounted  for 
_ percent  of  sales  revenue. 

34.5 

35.2 

m 

B 

2.  The  expense  of  operating  the  purchasing  function  was _ cent 

per  dollar  of  sales. 

0.2 

0.33 

.06-1 

20 

3.  It  costs _ cents  to  purchase  a  dollar  of  goods  or  services. 

0.75 

1.05 

.25-3 

21 

4.  There  was  one  purchasing  employee  for  every _ company 

employees. 

149 

226 

26-1,660 

19 

5.  There  was  one  purchasing  employee  for  each  $ _ million  of 

sales. 

34.9 

61.6 

7.8-538 

21 

6.  There  were  $ _ million  of  purchasing  department  purchases 

per  purchasing  employee. 

11.8 

11.4 

21 

7.  Each  professional  purchasing  employee  managed  $ _ million 

of  purchasing  department  purchases. 

18.1 

19.3 

3.7-47.9 

21 

8.  There  were _ active  suppliers  per  purchasing  employee. 

83 

89 

21  -297 

21 

9.  Minority-owned  businesses  received _ percent  of  total  pur¬ 

chase  dollars. 

2.2 

3.2 

1.1  -9.1 

21 

10.  Small  businesses  received _ percent  of  total  purchase  dol¬ 

lars. 

26.3 

24.3 

7-42 

8 

11.  Women-owned  businesses  received _ percent  of  total  pur¬ 

chase  dollars. 

2.6 

■a 

.9-4.4 

21 

12.  Electronic  Data  Interchange  (EDI)  was  used  to  process _ 

percent  of  company  purchase  dollars. 

5.1 

6.6 

.26-21 

21 

13.  The  purchasing  department  made _ percent  of  the  total  pur¬ 

chases. 

87.5 

85.8 

44-96.2 

20 

14.  After  a  purchase  request  was  received  in  the  purchasing  de¬ 
partment,  it  took _ days  for  a  supplier  to  receive  a  purchase  or¬ 

der. 

6.5 

11.4 

5.1  -25.5 

7 

NOTE:  The  first  numeric  column  contains  the  median  value  (i.e.,  the  point  where  half  the  industries  were  higher  and  half  lower) 
for  the  number  of  industries  represented,  as  shown  in  the  last  column.  The  mean  value  (the  sum  of  all  values  divided  by  the  number 
reporting)  and  the  range  of  values  (the  lowest  and  the  highest  values  reported)  for  each  benchmark  are  also  shown. 
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system  is  provided  by  the  Malcolm  Baldrige 
Award  criteria.  While  the  information  and 
analysis  area  is  only  one  of  the  factors  in  the 
Baldrige  assessment,  we  focus  on  it  here  be¬ 
cause  of  its  inherent  linkage  to  the  measurement 
system.  In  this  assessment,  three  factors  are 
considered: 

•  The  scope  and  management  of  quality  and 
performance  data  and  information  used  for 
planning,  day-to-day  management,  and 
evaluation  of  quality  and  operational  per¬ 
formance. 

•  Competitive  comparisons  and  benchmark¬ 
ing  to  support  improvement  of  quality  and 
operational  performance. 

•  Analysis  and  use  of  company-level  data 
related  to  quality,  customers,  and  opera¬ 
tional  performance. 

The  Baldrige  standard’s  emphasis  on  having  a 
sound,  systematic  procedure  is  shown  by  its 
goal  of  mobilizing  an  organization  towards 

•  a  sound,  systematic  prevention  procedure 
refined  by  means  of  evaluation  and  im¬ 
provement  cycles; 

•  a  system  integrated  into  all  major  business 
areas  and  all  support  areas; 

•  excellent  (world  class)  results  in  major  ar¬ 
eas  and  good-to-excellent  trends  in  all 
support  areas;  and 

•  producing  evidence  that  results  clearly  are 
caused  by  the  approach  used.  [17] 

We  find  these  considerations  to  be  important 
and  incorporate  them  in  the  design  of  the  meas¬ 
urement  system  set  forth  in  this  paper. 

Procurement  Executive  Association 

The  Procurement  Executive  Association  (PEA), 
composed  of  key  Federal  civil  agencies’  ac¬ 
quisition  officials,  established  a  Performance 
Measurement  Action  Team  to  assess  the  state  of 
the  current  system,  identify  innovative  ap¬ 


proaches  to  measuring  performance,  and  de¬ 
velop  strategies  and  recommendations  for  meas¬ 
uring  the  health  of  agency  acquisition  systems. 
[13]  The  team  members  developed  an  expanded 
and  enhanced  “balanced  scorecard”  [18]  includ¬ 
ing  14  measures  in  five  areas:  financial,  cus¬ 
tomer,  internal  business,  innovation  and  learning, 
and  employee  empowerment.  They  measure 
strength  in  these  areas  by  using  a  combination  of 
survey  results  and  workload  and  financial  data 
collected  by  existing  information  systems. 

For  example,  they  use  a  customer  survey  to 
determine  satisfaction  with  on-time  delivery, 
planning  activities  (including  establishment  of 
lead-times  and  acquisition  milestones),  and  on¬ 
going  communication.  Other  measures  are  the 
number  of  actions  per  employee,  purchasing 
department  costs  as  a  percentage  of  contract 
dollars  awarded,  and  the  ratio  of  dollars  saved 
or  costs  avoided  as  a  percentage  of  contract 
dollars  obligated.  While  this  model  has  been 
tested  at  selected  civil  agency  sites,  it  has  not 
been  fully  validated  across  the  entire  govern¬ 
ment  spectrum.  The  PEA  team  believes  that  the 
recently  completed  test  indicates  great  potential 
for  using  this  model  to  measure  the  health  of 
procurement  organizations.  In  at  least  one 
agency  it  is  being  used  to  replace  the  traditional 
procurement  management  review  program.  [19] 

We  believe  that  the  balanced  scorecard  ap¬ 
proach  as  applied  to  procurement  functions  is 
an  appropriate  response  to  including  multiple 
measures  in  a  single  performance  measurement 
model.  Trading  off  one  perspective  area  against 
another  and  motivating  superior  performance 
will  continue  to  be  a  challenge  as  the  balanced 
scorecard  is  applied  in  practice. 

Historical  Contracting  Measures 

Within  government  contracting  offices,  a  vari¬ 
ety  of  performance  measures  have  been  used 
over  the  years  to  measure  organizational  per¬ 
formance.  One  paper  on  procurement  perform¬ 
ance  led  to  the  development  of  the  performance 
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measures  covering  quality,  effectiveness,  and 
efficiency  that  are  set  forth  in  Exhibit  2.  [20] 
While  an  office  may  not  want  to  use  all  of  the 
measures  in  each  category,  selecting  a  few  of 
them  may  provide  a  start  toward  developing  a 
comprehensive  performance  measurement  sys¬ 
tem.  So  far,  however,  we  are  unaware  of  any 
effort  to  combine  the  measures  into  an  inclusive 
single  measure. 

Corporate  Model 

James  P.  Morgan  has  described  a  Purchasing 
Productivity  Index  that  combines  several  meas¬ 
ures.  [21]  The  first  is  the  proportion  of  total 
purchases  completed  through  preferred  meth¬ 
ods:  corporate  or  cooperative  divisional  agree¬ 
ments,  in-house  purchases,  and  competitive 
procurement.  The  second  is  the  average  of  the 
vendor  quality  rating  and  timeliness-of-delivery 
ratings.  The  third  is  the  profit  contribution,  cal¬ 
culated  as  the  total  of  net  savings,  cost  avoid¬ 
ance,  earned  discounts,  and  transportation  sav¬ 
ings  in  proportion  to  total  purchases.  These 
three  factors  are  divided  by  100  and  then  mul¬ 
tiplied  by  one-third  of  the  value  of  the  perform¬ 
ance  factor.  The  performance  factor  is  a  meas¬ 
ure  of  the  divisional  performance  as  a 
proportion  of  the  corporate  performance  in  dol¬ 
lars  per  fine  item,  dollars  per  person,  and  dollars 
per  buyer. 

Because  all  measurement  systems  motivate  be¬ 
havior,  the  goal  is  to  encourage  optimum  behav¬ 
ior  to  maximize  performance.  When  they 
combine  measures,  managers  must  take  great 
care  that  the  ranges  of  performance  and  weights 
applied  to  each  measure  lead  to  outcomes  that 
they  desire.  For  example,  in  the  just-described 
Purchasing  Productivity  Index,  since  profit 
contribution  seldom  could  be  expected  to  top  20 
cents  on  the  dollar,  a  purchasing  division  direc¬ 
tor  might  find  it  much  more  fruitful  to  concen¬ 
trate  either  on  preferred  methods  or  on  the  ven¬ 
dor’s  performance,  which  can  provide  much 
higher  scores.  If  this  outcome  is  acceptable  to 
the  upper  level  managers,  the  performance 


measurement  system  is  correctly  motivating  the 
division  director.  But  if  it  is  not  desired  that 
more  concentration  be  placed  on  preferred 
method  and  less  on  profit  contribution,  the  sys¬ 
tem  may  need  to  be  changed.  Before  imple¬ 
menting  any  measurement  system,  even  in  a 
pilot  application,  it  is  important  to  perform  a 
sensitivity  analysis,  as  discussed  later,  to  de¬ 
termine  the  true  incentives  that  will  be  gener¬ 
ated  by  that  system. 

Measurement  Baseline  Summary 

In  applying  performance  measures,  agencies 
must  consider  a  number  of  other  issues  as  well. 
One  of  them  is  that,  because  the  organization 
and  responsibilities  of  contracting  offices  may 
vary,  managers  cannot  compare  the  perform¬ 
ance  of  one  office  with  that  of  another  unless 
everyone  involved  understands  the  differences 
in  organizational  tasking,  accounting  systems, 
and  measurement  methodologies.  But  the  meas¬ 
ures  can  be  used  with  great  effectiveness  to  as¬ 
sess  the  performance  of  a  single  office  over  time. 

The  measures  for  quality,  effectiveness,  and 
efficiency  and  the  measurement  systems  that 
have  been  described  have  not  usually  been  co¬ 
ordinated  or  consolidated  into  a  measurement 
model  resulting  in  a  single  numerical  value  that 
provides  a  unified  measure  of  operational  con¬ 
tracting  cost,  output,  and  quality  together.  Uni¬ 
fying  such  measures,  as  we  describe  below, 
provides  organizations  with  a  yardstick  that  em¬ 
pirically  demonstrates  performance  over  time. 

PUC  APPLIED  TO  CONTRACTING 

Performance  unit  costing  (PUC)  considers  the 
cost  of  operations  relative  to  performance  units. 
Performance  units  represent  completed  contract 
actions,  adjusted  for  quality  by  an  index  measur¬ 
ing  such  aspects  as  customer  satisfaction  and 
attainment  of  goals  for  competition  or  socioeco¬ 
nomic  programs.  This  approach  trades  off  qual¬ 
ity,  output,  and  operational  cost  considerations. 
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Exhibit  2. 

Suggested  Performance  Measures 


Measure 

Description/Benchmark 

Quality 

Customer  satisfaction 

Evaluate  selected  items  from  customer  survey,  considering  both  positive 
and  negative  comments 

Solicitation 

Determine  the  quality  of  the  solicitation 

Proposals/competition 

Assess  procurement  publicity  and  attainment  of  competitive  goals 

Evaluation/negotiation 

Evaluate  the  professionalism  with  which  proposals  are  evaluated  on  both 
technical  and  business  bases  and  the  use  of  the  resulting  information  in 
negotiations 

Organizational  placement 

Assess  visibility  of  office;  should  be  equivalent  to  that  of  program  offices 
and  high  enough  to  command  management  attention 

Socioeconomic  goals 

Determine  extent  to  which  goals  are  achieved 

Training 

Determine  number  of  hours  spent  in  training  (e.g.,  40  per  person  per  year) 

Turnover 

Examine  turnover  exceeding  20  percent  per  year 

Management  information 

Evaluate  extent  and  usefulness  of  management  information  systems 

Effectiveness 

Customer  satisfaction 

Evaluate  selected  items  from  customer  survey,  considering  both  positive 
and  negative  comments 

Processing  time 

Determine  length  of  procurement  administrative  lead-time  and  success  in 
meeting  processing  time  goals 

Consolidation  of  orders 

Assess  extent  to  which  similar  programs  are  combined  to  rationalize  proc¬ 
essing 

Obligations 

Consider  degree  to  which  time  goals  for  obligating  funds  are  met 

Workload 

Evaluate  level  and  uniformity 

Delinquent  deliverables 

Calculate  percentage;  should  not  exceed  20  percent 

Closeout 

Determine  contract  closeout  backlog;  should  not  exceed  the  annual  num¬ 
ber  of  awards 

Efficiency 

Total  number  of  contracts 
awarded 

Count  the  number  of  contracts  awarded  per  year 

Number  of  contracts  awarded 
per  specialist 

Divide  the  annual  count  by  the  number  of  contract  specialists 

Total  value 

Total  the  dollar  amount  of  contracts  awarded  annually 

Value  of  contracts  awarded  per 

Divide  the  summation  of  annual  dollar  amount  by  the  number  of  contract 

specialist 

specialists 

Total  cost  of  contracts  operation 

Total  the  annual  operating  cost  (direct  and  indirect) 

Cost  of  contracts  operation  as  a 
percent  of  total  award  value 

Divide  the  annual  operating  cost  by  the  summation  of  annual  dollar  amount 

Cost  of  contracts  operation  as  a 
percent  of  agency  budget 

Divide  the  annual  operating  cost  by  the  annual  agency  budget 

Source:  [20] 
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The  most  difficult  task  when  developing  a  PUC 
measurement  approach  is  selecting  the  quality 
index  factors.  One  method  we  suggest  is  to  as¬ 
semble  a  group  of  customers  and  ask  them  to 
define  the  factors  they  believe  essential  for 
quality. 

In  earlier  work,  the  authors,  together  with  others 
at  LMI,  developed  a  PUC  contracting  quality 
index  by  assembling  a  team  of  highly  experi¬ 
enced  procurement  management  professionals 
and  having  the  team  jointly  list  desired  items 
pertaining  to  the  procurement  process,  as  seen 
by  principal  customers,  including  managers  of 
the  process.  From  this  long  list  of  desirable 
factors,  the  team  members  then  individually 
ranked  the  importance  of  the  listed  items  by  as¬ 
signing  either  3,  5,  7,  or  9  points  to  each  item. 
The  combined  result — the  average  for  each 
item  —  was  used  to  create  a  list  of  the  key  items 
(or  “quality  factors”),  that  indicates  how  they 
might  be  measured  and  the  relative  weights  as¬ 
signed  to  them  by  the  team. 

Exhibit  3  is  an  illustration  of  a  set  of  quality 
factors  and  weights  developed  by  the  authors  as 
an  example  to  be  used  for  this  paper.  In  this 
example,  customer  satisfaction  would  be  meas¬ 
ured  through  the  use  of  surveys.  We  think  that 
a  short  postcard-type  survey  should  be  used, 
requesting  information  on  issues  such  as  timeli¬ 
ness,  value  received  compared  to  funds  ex¬ 

Exhibit  3. 

PUC  Contracting  Quality  Factors  —  Example 


pended,  receipt  of  the  correct  product  or  serv¬ 
ice,  and  professionalism  of  the  contracting  staff. 
This  survey  would  be  completed  at  the  time  of 
purchase  request  receipt  (only  those  items  ap¬ 
plicable  to  planning),  at  the  time  of  contract 
award,  and  at  the  time  of  delivery.  We  include 
a  measure  related  to  increasing  the  use  of  com¬ 
mercial  sources  as  an  additional  measure  in  the 
example.  Finally,  we  have  included  the  attain¬ 
ment  of  socioeconomic  program  and  competi¬ 
tion  goals  as  example  measures. 

Once  the  quality  factors  and  their  weights  are 
established  to  the  satisfaction  of  the  customers, 
work  force,  and  managers,  they  can  be  used  to 
develop  the  PUC  model  for  the  activity.  This 
model  combines  the  quality  factors  with  the  op¬ 
erational  cost  of  the  work  unit  and  the  total 
number  of  completed  output  units. 

In  establishing  the  relationship  between  cost, 
output  units,  and  quality,  the  manager  must 
carefully  consider  the  emphasis  to  be  placed  on 
each  area  and  how  the  factors  will  relate  to  each 
other.  If  the  formula  is  simply  the  cost  (C)  di¬ 
vided  by  the  output  units  (O)  multiplied  by  the 
quality  index  (I)  [performance  unit  cost  = 
C/(OxI)],  any  proportional  change  in  one  factor 
will  have  the  same  impact  on  the  performance 
unit  cost  as  any  similar  change  in  another  factor. 
However,  if  we  want  to  put  additional  emphasis 
on  quality,  we  can  increase  the  importance  of 


Quality  Factor 

Measurement 

Index  Weight 

Customer  satisfaction  — timeliness,  value,  cor¬ 
rect  product/service,  contracting  professionalism 

Customer  survey 

0.60 

Increasing  use  of  commercial  sources 

Percentage  of  proposals  received  com¬ 
pared  to  sources  solicited 

0.20 

Attainment  of  socioeconomic  program  goals 

Percentage  reflecting  degree  to  which 
goals  are  achieved  or  criteria  satisfied 

0.10 

Attainment  of  competition  goals 

Percentage  reflecting  degree  to  which 
goals  are  achieved  or  criteria  satisfied 

0.10 

Total 

1.00 
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that  factor  in  our  equation  by  raising  it  to  a 
power,  say  by  squaring  it  [performance  unit  cost 
=  C/(OxI2)].  This  is  the  example  used  in  the 
balance  of  this  paper.  (Some  examples  of  the 
application  of  the  PUC  approach  with  this 
added  quality  emphasis  are  shown  in  Exhibit  4.) 

The  situation  in  Column  A  of  Exhibit  4  shows 
low  output  (900  output  units)  and  poor  quality 
(a  quality  index  rating  of  0.60  out  of  1.00),  re¬ 
sulting  in  a  total  of  324  performance  units.  This 
situation  results  in  a  performance  unit  cost  of 
$15,432  each.  Holding  total  organizational  cost 
constant  while  improving  both  quality  and  out¬ 
put,  as  in  the  example  in  Column  B,  reduces  the 
per  unit  performance  cost  to  $6,457.  Finally, 
the  example  in  Column  C  shows  the  change 
resulting  from  improving  the  quality  index  to  its 
highest  level  while  holding  all  other  factors 
constant,  resulting  in  a  performance  unit  cost  of 
$5,000.  In  these  examples,  producing  addi¬ 
tional  output  and  increasing  the  quality  index 
achieves  lower  performance  unit  costs.  Manag¬ 
ers  may  also  affect  the  other  item  in  the  equa¬ 
tion —  cost  —  which  would  also  change  the 
PUC  results. 

So  far  in  this  discussion,  “contract  action”  has 
been  the  output  unit  measure.  Depending  on 
the  office  being  measured,  this  term  could  take 
on  a  variety  of  meanings.  In  some  situations,  an 
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Performance  Unit  Costing  —  Examples 


office  might  choose  to  measure  only  initial 
contract  awards.  However,  most  contracting 
offices  would  choose  to  add  some  of  the  follow¬ 
ing:  delivery  orders,  option  exercises,  modifi¬ 
cations  (funded,  unfunded,  and  administrative), 
terminations,  basic  ordering  agreement  awards, 
and  closeouts.  These  items  would  usually  be 
given  various  weights  based  on  their  perceived 
level  of  difficulty.  Initial  contract  awards  are 
also  often  categorized  by  dollar  value,  type  of 
contract,  solicitation  method,  or  some  other 
measure  of  complexity.  The  exact  definition  of 
“contract  action”  should  be  determined  by  the 
office  being  measured.  But  whatever  the  defi¬ 
nition,  it  must  be  thoughtfully  selected  so  as  to 
reflect  the  incentives  desired  by  management. 

The  PUC  value  of  an  organization  may  be  af¬ 
fected  by  outside  variables  as  well  as  by  actions 
taken  by  the  managers  or  other  members  of  the 
organization.  For  example,  if  the  office’s 
workload  is  reduced  through  downsizing  or  for 
any  other  reason,  unit  costs  will  initially  in¬ 
crease  as  a  certain  amount  of  fixed  overhead 
cost  is  spread  over  fewer  output  units.  In  the 
long  run,  short-term  fixed  costs  are  also  variable 
and  can  be  affected  by  management  action.  If 
such  outside  forces  affect  the  organization  dur¬ 
ing  a  measurement  period,  they  must  be  taken 
into  consideration  separately. 


Factors 

Possible  Outcomes 

A 

Low  Output 

Poor  Quality 

B 

High  Output 

Better  Quality 

c 

High  Output 
Perfect  Quality 

Output  units  (O) 

900 

1,000 

1,000 

Quality  index  (1) 

0.60 

0.88 

1.00 

Performance  units  (PU)  =  (O  x  I2) 

324 

774 

1,000 

Cost  (C) 

$5,000,000 

$5,000,000 

$5,000,000 

Performance  unit  cost  =  C/PU 

$15,432 

$6,457 

$5,000 
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Exhibit  5. 

Sensitivity  Examples 


Factors 

Baseline 

Example  1 

Example  2 

Example  3 

Output  units  (0) 

1,000 

850 

1,000 

1,000 

Quality  index  (1) 

1.00 

1.00 

0.85 

1.00 

Performance  units  (PU)  =  (0  x  l2) 

1,000 

850 

722.5 

1,000 

Cost  (C) 

$5,000,000 

$4,000,000 

$4,000,000 

$4,000,000 

Performance  unit  cost  [PUC  =  C/(0  x  I2)] 

$5,000 

$4,705.88 

$5,536.33 

$4,000 

Unit  savings  (loss) 

(baseline  performance  unit  cost  minus 
example  performance  unit  cost) 

$0 

$294.12 

($536.33) 

$1,000 

Total  savings  (loss)  (unit  savings  x  PU) 

$0 

$250,002 

($387,498.43) 

$1,000,000 

Gainsharing  (50%  of  total  savings) 

$0 

$125,001 

$0 

$500,000 

COMBINING  COST  AND  QUALITY  — 
PUC  SENSITIVITY  ANALYSIS 

Once  the  PUC  formula  is  developed  and  the 
quality  factors  and  relative  weights  are  agreed 
upon,  a  combination  for  considering  both  cost 
and  quality  will  be  in  place  to  measure  and  mo¬ 
tivate  staff  performance.  It  is  essential  that 
managers  understand  the  dynamic  forces  that 
this  combination  may  produce  and  the  effect 
they  have  on  motivation,  to  be  sure  that  they  are 
consistent  with  organizational  objectives.  For 
this  reason,  before  any  new  measurement  sys¬ 
tem  is  prototyped,  a  sensitivity  analysis  should 
be  conducted  to  ensure  that  the  system’s  ele¬ 
ments  will  work  synergistically  to  improve  or¬ 
ganizational  performance.  Also,  if  the  system  is 
to  be  most  effective,  the  staff  members  must 
find  that  there  is  something  in  it  for  them. 
Whether  this  “something”  is  a  gainsharing  pro¬ 
gram,1  or  some  other  reward  (as  encouraged  by 
the  Federal  Acquisition  Streamlining  Act  of 
1994  [22]),  the  basis  for  awarding  it  will  be  a 


'  Gainsharing  programs  are  used  to  motivate  team 
members  by  providing  them  financial  awards  for  im¬ 
proved  performance.  Although  known  by  many  names 
and  having  infinite  variety  in  the  details  of  their  appli¬ 
cations,  they  typically  take  a  proportion  (say,  50  per¬ 
cent)  of  the  team’s  operational  performance  savings 
and  distribute  it  among  the  team  members. 


lower  performance  unit  cost.  (Although  the 
motivational  effects  of  gainsharing  remain  con¬ 
troversial,  many  people  think  that  the  ability  to 
receive  a  portion  of  the  savings  will  encourage  a 
great  deal  of  innovation  among  work  teams. 
This  paper  assumes  that  such  a  program  is  in 
existence.) 

Under  this  measurement  system,  three  variables 
are  considered  simultaneously:  the  number  of 
units  produced,  the  quality  index,  and  the  total 
cost.  Because  quality  at  any  cost  is  not  realistic 
in  these  restrictive  budget  times,  a  balance  is 
necessary.  Playing  out  these  and  other  alterna¬ 
tives  may  identify  less-than-optimal  results.  If 
managers  of  a  procurement  office  are  not  satis¬ 
fied  with  the  measurement  scenarios,  they 
should  change  the  weights  used  in  the  PUC 
formula,  in  order  to  encourage  the  desired  be¬ 
havior.  Managers  can  change  the  emphasis  by 
changing  the  weights  in  Exhibit  3  so  that  an 
area  they  value  more  highly  has  more  weight. 
Or  they  can  consider  other  factors.  To  deter¬ 
mine  the  formula’s  motivational  aspects,  it  is 
important  for  managers  to  test  it  under  various 
scenarios.  Three  examples  are  shown  in  Exhibit  5. 

As  can  be  seen  from  the  exhibit,  the  level  of 
quality  is  very  significant.  Any  change  in  value 
of  the  quality  index  (since  it  is  squared  in  the 
formula)  makes  a  large  change  in  the  perform- 


414 


ance  unit  cost.  With  the  formula  used  here, 
even  with  a  20  percent  reduction  in  the  operat¬ 
ing  cost  of  the  office,  and  assuming  mainte¬ 
nance  of  the  same  output,  quality  cannot  drop 
by  more  than  10  percent  to  maintain  nearly  the 
same  PUC  value  {$4,000,000/[l,000x(.90)2]  = 
$4,938.27}. 

Since  the  quality  index  is  a  key  variable,  an¬ 
other  area  that  managers  must  review  is  the  de¬ 
termination  of  the  index  weights  themselves. 
Given  the  information  in  Exhibit  3,  meeting  the 
customer  satisfaction  factor  is  three  times  as 
important  as  increasing  the  use  of  commercial 
sources  and  she  times  as  important  as  attaining 
goals  in  either  socioeconomic  areas  or  competi¬ 
tion.  Exhibit  6,  using  the  same  factors  as  Ex¬ 
hibit  3,  shows  how  measurement  tradeoffs  af¬ 
fect  the  quality  index. 

To  provide  the  basic  illustration  of  the  quality 
index,  Column  A  of  Exhibit  6  shows  the  ideal 
case,  with  each  measure  scored  at  100  percent, 
resulting  in  a  total  score  of  1.00.  Column  B  is 
the  same  example,  but  with  each  factor  scored 
at  90  percent,  resulting  in  a  score  of  .90.  Col¬ 
umns  C  and  D  achieve  the  same  score,  but  in 
different  ways.  The  example  shown  in  Column 
C  indicates  a  poorer  job  of  achieving  customer 
satisfaction  goals,  while  Column  D  shows  a 
potential  problem  in  meeting  socioeconomic 
and  competition  goals.  To  you  as  a  manager, 

Exhibit  6. 

Quality  Factors  with  Alternative  Quality  Outcomes 


are  the  examples  in  Columns  B,  C,  and  D  all 
equally  pleasing?  If  not,  you  need  to  reevaluate 
the  weights  for  each  area. 

Finally,  the  staff  member  working  with  the 
situation  in  Column  E,  which  may  be  the  most 
realistic,  must  decide  what  actions  to  take  to 
most  improve  the  score  for  the  office  (and  that 
person’s  reward  as  a  participant  in  the  team’s 
gainsharing  program).  This  will  involve  de¬ 
termining  the  tradeoff  between  the  number  of 
points  available  in  each  category  versus  the  dif¬ 
ficulty  in  achieving  them.  It  may  be  easier  to 
attempt  to  achieve  the  0.1  additional  points  by 
increasing  the  use  of  commercial  sources  than  it 
is  to  earn  0.09  points  by  making  all  customers 
satisfied.  Managers  must  be  confident  that  the 
employee’s  choice  will  be  the  one  the  managers 
desire. 

Another  concern  with  any  measurement  system 
is  a  change  in  the  staffs  behavior  that  may  re¬ 
sult  from  collection  of  a  particular  data  element. 
For  example,  would  a  contract  specialist  attempt 
to  limit  the  sources  sought  to  known  previous 
offerors  in  order  to  increase  the  proportion  of 
commercial  proposals?  If  so,  that  measure  may 
be  poorly  designed  and  a  correction  would  be 
needed.  Adding  another  measure,  such  as  con¬ 
sidering  the  total  number  of  new  sources 
(previously  not  doing  business  with  the  gov¬ 
ernment)  solicited,  as  compared  to  the  number 


Socioeconomic  goals  met 

0.10 

1.00 

1.00 

0.09 

0.50 

0.05 

0.90 

Competition  goals  met 

0.10 

1.00 

0.10 

0.90 

0.09 

1.00 

0.09 

0.50 

0.05 

0.90 

E 


Value 


0.51 

0.10 

0.09 


Total 

1.00 

1.00 

0.90 

0.91 

0.90 

0.79 

of  proposals  received  from  such  companies, 
may  correct  the  otherwise  counterproductive 
behavior. 

DATA  COLLECTION 

To  institute  a  measurement  system,  certain  data 
must  be  collected,  usually  on  a  monthly  basis. 
To  apply  the  PUC  method  described  above,  the 
manager  must  know  the 

•  customer  satisfaction  survey  data,  as  de¬ 
scribed  above; 

•  contracting  operation’s  total  cost,  includ¬ 
ing  indirect  costs; 

•  number  of  contract  actions  awarded; 

•  socioeconomic  program  goals; 

•  degree  of  attainment  of  the  socioeconomic 
program  goals; 

•  competition  goals;  and 

•  degree  of  attainment  of  the  competition 
goals. 

It  might  be  assumed  that  any  contract  action 
will  be  treated  as  having  the  same  value  as  any 
other  contract  action.  But,  unchecked,  this  pro¬ 
cedure  may  lead  contract  specialists  to  execute  a 
large  number  of  unfunded  administrative  modi¬ 
fications.  To  avoid  such  a  result,  the  contract 
manager  may  establish  categories  of  contract 
actions  to  provide  additional  weight  to  those 
perceived  to  add  the  most  value  to  the  opera¬ 
tion.  If  this  is  done,  it  multiplies  the  data  col¬ 
lection  effort  accordingly,  since  data  must  be 
collected  for  each  category. 

If  other  items  from  Exhibit  1  or  2  are  desired  for 
part  of  the  contracting  office  measurement  sys¬ 
tem,  data  collection  requirements  must  be  ad¬ 
justed  accordingly. 

Finally,  it  is  very  important  that  the  measures  be 
consistent  from  period  to  period.  For  example, 
a  change  in  the  calculation  of  charges  for  the 


square  footage  occupied  by  the  contracting  of¬ 
fice  (when  there  has  been  no  change  in  the  size 
of  the  office  itself)  would  cause  a  change  in  the 
PUC  value  that  did  not  reflect  a  change  in  the 
office’s  efficiency  of  operations.  By  holding 
the  definition  of  the  measures  constant,  or  by 
carefully  adjusting  the  PUC  value  on  the  basis 
of  known  differences,  year-to-year  progress  can 
be  evaluated.  As  noted  above,  definitional  and 
operational  differences  usually  mean  that  the 
PUC  values  achieved  by  offices  in  different 
agencies  may  not  be  able  to  be  compared  with 
one  another. 

CONCLUSION 

The  PUC  concept  can  be  useful  for  contracting 
offices  to  combine  the  factors  of  quality,  output, 
and  cost.  The  ultimate  outcome  of  a  single 
PUC  value  helps  the  office  to  track  its  perform¬ 
ance  over  time.  It  can  help  managers  develop  a 
procurement  measurement  system  that  will 
provide  those  incentives  to  the  work  force  that 
management  wishes  to  promote. 

The  examples  in  the  paper  show  that  all  ele¬ 
ments  of  the  measurement  system  are  important 
and  must  be  thought  through  carefully  before 
being  introduced.  These  elements  include  the 

•  items  to  be  measured  (e.g.,  customer  satis¬ 
faction  or  attaining  competition  goals); 

•  definitions  of  the  customer  and  of  each 
item  to  be  measured  (e.g.,  contract  ac¬ 
tions); 

•  weight  applied  to  each  item  in  the  quality 
index  (or  any  index); 

•  weight  applied  to  each  item  in  the  PUC 
formula  (the  degree  of  importance  attrib¬ 
uted  to  each  item); 

•  gainsharing  payout  (or  other  reward)  cal¬ 
culation  for  improved  performance;  and 
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•  ability  to  accurately  record  the  original 
data. 

The  importance  of  considering  carefully  all 
these  elements  is  great  regardless  of  the  meas¬ 
urement  method  used  —  not  just  for  PUC  — 
and  it  is  imperative  that  a  sensitivity  analysis  be 
performed  to  determine  just  what  incentives 
will  be  created  by  any  proposed  system  of 
measurement. 
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REASONS  FOR  COST  AND  SCHEDULE  GROWTH 


by  Karen  Windau  Tyson 
Institute  for  Defense  Analyses 


ABSTRACT 

Programs  to  acquire  major  weapon  systems 
often  end  up  taking  longer  and  costing  more 
than  planned.  Delays  in  schedules  may  leave 
U.S.  forces  without  needed  capabilities  and 
vulnerable  to  enemy  weapons,  if  the  system 
is  not  completed  on  time.  Delays  often 
translate  into  higher  cost,  which  creates 
further  problems.  Weapon  system  costs  have 
been  growing  both  from  generation  to 
generation,  and,  within  a  given  system,  from 
plan  to  realization. 

This  paper  highlights  cost  and  schedule 
experience  of  programs  to  acquire  tactical 
missiles  and  tactical  aircraft  since  the  1960s. 
By  looking  at  past  programs,  the  paper 
examines  hypotheses  ahout  .what  separates 
the  kinds  of  programs  completed  on  schedule 
and  within  cost  plans  from  those  that  suffer 
from  schedule  and  cost  growth. 

On  average,  the  total  program  cost  growth 
for  tactical  missile  programs  is  substantially 
higher  than  for  tactical  aircraft,  and 
development  cost  growth  and  development 
schedule  growth  are  also  high. 

The  histories  of  the  tactical  missile  programs 
were  examined  to  determine  the  reasons  for 
schedule  and  cost  growth.  Keys  to 
preventing  schedule  growth  in  development 
are  technical  realism  and  willingness  to  make 
tradeoffs.  Programs  with  high  development 
schedule  growth  tended  to  underestimate 
technical  difficulty 

Cost  and  schedule  growth  measures  were 
also  calculated  for  a  sample  of  seven  tactical 


aircraft.  These  measures  are  less  dispersed 
than  those  for  tactical  missiles.  The  aircraft 
programs  tended  to  receive  more 
management  attention  and  more  protection 
from  production  schedule  stretch  than  the 
tactical  missiles. 

The  case  analyses  indicate  that  there  are 
many  useful  lessons  to  be  gleaned  from 
historical  perspective,  lessons  that  cannot 
always  be  captured  in  a  quantitative 
estimating  relationship.  Despite  individual 
differences  in  programs,  the  importance  of 
understanding  the  level  of  technical  difficulty 
when  original  schedule  and  cost  estimates 
are  made,  of  strict  phasing  and  vigorous 
testing,  and  of  adhering  to  production  plans 
are  home  out  by  analysis  of  past  strategies 
and  outcomes. 

A.  INTRODUCTION 

Programs  to  acquire  major  weapon  systems 
usually  cannot  follow  their  carefully-laid 
acquisition  plans  in  every  detail.  They  often 
end  up  taking  longer  and  costing  more  than 
planned. 

The  time  required  to  design  and  develop 
major  weapon  systems  has  apparently 
lengthened,  and  weapon  systems  often 
exceed  their  planned  development  schedules. 
A  1990  RAND  Corporation  study  found  that 
programs  from  the  1970s  and  1980s  took 
longer  than  those  from  the  1950s  and  1960s. 
The  demonstration/validation  phase  (from 
Milestone  I  to  Milestone  II  in  the  process 
required  by  DoD  regulations)  and  the 
engineering  and  manufacturing  development 
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phase  (measured  from  Milestone  II  to  the 
first  delivery)  each  take  about  a  year  longer 
than  they  used  to  [1].  A  1992  IDA  study 
found  that,  on  average,  systems  take  one- 
third  longer  than  planned  to  progress  from 
Milestone  II  to  achievement  of  initial 
operational  capability  (IOC)  [2], 

Excessive  schedules  have  two  significant 
negative  effects.  One  is  that  delays  in 
schedules  may  leave  U.S.  forces  without 
needed  capabilities  and  vulnerable  to  enemy 
weapons,  if  the  system  is  not  completed  on 
time.  The  other  is  that  delays  often  translate 
into  higher  cost,  which  creates  further 
problems. 

Weapon  system  costs  have  been  growing 
both  from  generation  to  generation,  and, 
within  a  given  system,  from  plan  to 
realization.  Growth  in  the  cost  of  weapon 
systems  appears  to  be  a  chronic  problem.  Of 
82  programs  examined  by  IDA  in  1992,  66 
had  experienced  cost  growth  [2], 

The  defense  acquisition  community  is 
concerned  that  programs  to  develop  major 
weapon  systems  take  too  long  and  cost  too 
much.  In  1986,  the  President's  Blue  Ribbon 
Commission  on  Defense  Management  (the 
Packard  Commission)  called  schedule  length 
"a  central  problem  from  which  most  other 
acquisition  problems  stem"  [3],  Cost  growth 
forces  the  DoD  to  revise  budget  plans, 
makes  systems  less  affordable,  and  frequently 
erodes  congressional  support  for  acquisition 
programs. 

This  paper  highlights  cost  and  schedule 

experience  of  programs  to  acquire  systems  in 
two  major  product  categories,  tactical 

missiles  and  tactical  aircraft.  Tactical  missiles 

are  particularly  prone  to  cost  and  schedule 
difficulties,  as  prior  research  has  shown  [2,  4, 
and  5],  Tactical  missiles,  while 


technologically  complex,  are  viewed  as  being 
less  glamorous  than  tactical  aircraft,  and 
missile  programs  often  do  not  receive 
priority.  Tactical  aircraft  are  among  the  most 
technologically  complex  products  the 
military  buys.  They  are  developed  on  a 
custom  basis,  usually  pushing  the  state  of  the 
art.  Tactical  aircraft  development  is 
constrained  by  size  and  weight  restrictions, 
and  is  spurred  toward  ever-faster  goals  for 
operating  speeds.  Moreover,  the  physical 
environment  in  which  these  aircraft  operate 
often  involves  extreme  temperatures,  high 
vibration,  and  G-forces  [6], 

By  looking  at  past  programs,  the  paper 
examines  hypotheses  about  what  separates 
the  kinds  of  programs  completed  on  schedule 
and  within  cost  plans  from  those  that  suffer 
from  schedule  and  cost  growth.  On  average, 
the  total  program  cost  growth  for  tactical 
missile  programs  is  substantially  higher  than 
for  other  equipment  types,  while 
development  cost  growth  and  development 
schedule  growth  are  among  the  highest. 
Development  schedule  growth  for  tactical 
missile  programs  is  high. 

B.  DATA 

The  sample  consisted  of  fifteen  tactical 
missiles  and  seven  tactical  aircraft.  None  of 
the  programs  in  the  sample  was  canceled. 
The  data  on  these  systems  were  obtained 
from  Selected  Acquisition  Reports  (SARs), 
from  historical  memoranda  to  support  DoD 
program  reviews,  and  from  summaries  of 
program  data  [7  and  8],  Data  were  current  as 
of  the  1992  SARs.1 


1  Only  the  first  version  of  each  system  was 
examined  and  not  subsequent  modifications, 
unless  they  were  considered  as  separate 
programs.  DoD  treatment  of  modification 
programs  in  its  reporting  and  review  process 
has  not  always  been  uniform.  Sometimes,  as  in 
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Cost  and  schedule  growth  were  examined  by 
comparing  actual  outcomes  with  those 
planned  at  the  Milestone  II  review,  when  the 
commitment  for  engineering  and 
manufacturing  development  was  made. 
Outcomes  were  expressed  as  a  ratio  of  actual 
to  planned  cost  or  schedule. 

The  development  cost  growth  (DCG)  ratio  is 
defined  as: 

DCG  =  ADC/PDC, 

where 

ADC  =  actual 

development  cost,  the  actual  cost  to  develop 

the  system,  measured  in  millions  of  FY  1994 

dollars  from  Milestone  II  to  the  end  of 
development  of  the  first  version;  and 

PDC  =  planned 

development  cost,  the  planned  cost  to 
develop  the  system,  measured  in  millions  of 
FY  1994  dollars  from  Milestone  II  to  the  end 
of  development  of  the  first  version. 

The  production  cost  growth  (PCG)  ratio  is 
defined  as: 

PCG  =  APC/PPC, 

where 

APC  =  actual 

production  cost,  the  actual  cost  to  produce 
the  planned  quantity  of  the  system,  measured 
in  millions  ofFY  1994  dollars;  and 

PPC  =  planned 

production  cost;  the  planned  cost  at 
Milestone  II  to  produce  the  planned  quantity 
of  the  system,  measured  in  millions  of  FY 
1994  dollars. 


the  case  of  the  F-14,  a  new  version  was  treated 
as  a  new  program  and  went  through  a  new  set  of 
reviews  and  had  separate  documentation.  Other 
times,  as  in  the  case  of  the  F-15,  DoD  treated 
the  program  as  one,  despite  the  technical 
differences  between  the  F-15A/B  and  the  F- 
15E.  When  a  data  source  included  costs  for 
modified  versions,  data  on  production  schedules 
were  used  to  obtain  the  actual  costs  for  the  first 
version. 


If  the  actual  quantity  produced  was  greater 
than  that  planned  at  Milestone  II,  the 
numerator  was  the  cost  to  produce  only  the 
planned  quantity.  If  the  actual  quantity  was 
less  than  that  planned,  a  price-improvement 
curve  was  used  to  project  the  cost  of 
producing  the  planned  quantity.  In  every 
case,  only  data  on  actual  costs  were 
included.  DoD  future  planned  costs  were  not 
used  to  measure  cost  growth. 


The  total  program  cost  growth  (TPCG)  ratio 
is  defined  as: 


TPCG  =  (ADC  +  APC)/(PDC  +  PPC). 


Schedule  measures  were  calculated  similarly. 
The  development  schedule  growth  (DSG) 
ratio  is  defined  as: 

DSG  =  ADS/PDS, 

where 


ADS  =  actual 

development  schedule,  actual  time  to 
develop  the  first  version  of  the  system, 
measured  in  months  from  Milestone  II  to 
initial  operational  capability  (IOC);  and 
PDS  =  planned 

development  schedule,  planned  time  to 
develop  the  first  version  of  the  system, 
measured  in  months  from  Milestone  II  to 
IOC. 


The  production  schedule  stretch  (PSS)  ratio 
is  defined  as: 

PSS  =  APS/PPS, 

where 

APS  =  actual 

production  schedule,  actual  time  to  produce 
the  planned  quantity  of  the  system,  measured 
in  months  from  Milestone  III  to  the  end  of 
production  of  the  planned  quantity;  and 
PPS  =  planned 

production  schedule,  planned  time  to 
produce  the  planned  quantity  of  the  system, 
measured  in  months  from  Milestone  III  to 
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the  end  of  production  of  the  planned 
quantity. 

The  timing  of  production  is  based  on 
obligation  of  funds,  not  deliveries,  because 
that  is  the  way  the  plans  are  made.  If  the 
planned  quantity  has  not  yet  been  produced, 
a  linear  projection  was  made  based  on 
actuals. 

Total  schedule  growth  (TSG)  ratio  is  defined 
as  follows: 

TSG  =  (ADS+APS)/(PDS+PPS). 

TPCG  and  TSG  may  be  useful  in 
understanding  the  effect  of  development 
disruptions  or  successes  on  the  total 
program.  For  example,  we  may  observe  that 


a  program  completed  development  seriously 
behind  schedule,  but  within  its  planned  cost. 
While  it  is  possible  at  that  juncture  to  have  a 
production  program  proceed  as  planned,  it  is 
also  possible  that  issues  of  performance, 
producibility,  or  maintainability  are  not 
completely  resolved  until  after  production  is 
underway,  adding  to  production  costs. 

Table  1  presents  cost  and  schedule  growth 
measures  for  the  sample  of  fifteen  tactical 
missiles.  The  programs  for  which 
information  could  be  obtained  represent  a 
span  of  twenty  years,  with  FSD  start  dates 
ranging  from  1962  to  1982.  The  sample 
includes  tactical  missiles  from  the  Army,  the 
Navy,  and  the  Air  Force. 


Table  1.  Cost  and  Schedule  Growth  Measures  for  Tactical  Missiles 


Development  _ Production _  Total  Program 


Missile 

Cost 

Growth 

Schedule 

Growth 

Cost 

Growth 

Schedule 

Stretch 

Cost 

Growth 

Schedule 

Stretch 

Phoenix  AIM-54A 

1.54 

1.94 

1.35 

1.03 

1.38 

1.38 

AMRAAM 

1.76 

2.29 

1.84 

2.73 

1.84 

2.61 

Hellfire 

1.22 

1.44 

1.60 

0.99 

1.46 

1.23 

HARM 

1.61 

1.59 

1.51 

1.17 

1.52 

1.33 

Sparrow  AIM-7F 

4.26 

2.80 

1.58 

0.75 

1.73 

1.57 

TOW 

1.20 

1.45 

1.78 

3.90 

1.71 

2.78 

Sidewinder  AIM-9L 

4.89 

2.48 

2.07 

2.29 

2.23 

2.38 

TOW  2 

1.39 

1.00 

0.95 

1.02 

0.97 

1.01 

Harpoon 

0.90 

1.35 

1.85 

1.66 

1.66 

1.56 

Maverick  AGM-65D/G 

1.07 

1.98 

1.45 

1.93 

1.42 

1.95 

Sparrow  AIM-7M 

0.96 

1.49 

2.04 

1.08 

2.00 

1.26 

Sidewinder  AIM-9M 

2.04 

1.01 

1.02 

0.65 

1.10 

0.86 

Phoenix  AIM-54C 

1.67 

1.46 

1.93 

0.74 

1.89 

1.10 

Maverick  AGM-65A 

1.04 

1.33 

0.99 

0.98 

1.01 

1.09 

MLRS 

1.02 

1.06 

0.88 

1.06 

0.90 

1.06 

Median 

1.39 

1.46 

1.58 

1.06 

1.52 

1.33 

These  measures  showed  a  great  deal  of 
variability  among  the  20  programs  examined. 
Programs  took  from  50  months  to  134 
months  from  Milestone  II  to  IOC.  Only  one 


of  the  tactical  missile  programs  was  finished 
on  time.  The  program  with  the  highest 
development  schedule  growth  exceeded  its 
plan  by  1 80  percent.  Two  programs  came  in 
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under  budget,  while  two  others  doubled  in 
cost  from  their  Milestone  II  plan. 

Table  2  presents  cost  and  schedule  growth 
measures  for  the  sample  of  seven  tactical 
aircraft.  The  sample  included  the  major 
fighter  aircraft  programs  over  the  past  three 
decades.  The  F-14D  was  omitted,  because  its 
experience  was  much  more  limited  than  the 
others,  and  in  1992,  it  was  on  the  verge  of 


cancellation.  These  measures  are  less 
dispersed  than  those  for  tactical  missiles. 
Development  programs  had  fairly  small 
schedule  overruns,  with  the  F-16  finishing  in 
the  time  allotted  and  the  program  with  the 
most  schedule  growth,  the  F-14A,  running 
over  by  only  18  percent.  The  highest  cost 
growth  index  for  tactical  aircraft  was  1.40 
(for  the  F/A-18),  versus  2.23  for  the  tactical 
missiles. 


Table  2.  Cost  and  Schedule  Growth  Measures  for  Tactical  Aircraft 


Development 

Production 

Total  Program 

Cost 

Schedule 

Cost 

Schedule 

Cost 

Schedule 

Aircraft 

Growth 

Growth 

Growth 

Stretch 

Growth 

Stretch 

F-5E 

1.05 

1.06 

0.79 

0.70 

0.91 

0.83 

AV-8B 

1.40 

1.03 

0.93 

1.94 

0.98 

1.50 

F-16 

1.20 

1.00 

1.08 

1.11 

1.11 

1.06 

F-14A 

1.53 

1.18 

1.17 

2.08 

1.26 

1.69 

F-15A/B 

1.08 

1.05 

1.19 

1.14 

1.20 

1.10 

A-10 

1.37 

1.09 

1.34 

1.73 

1.37 

1.44 

F/A-18 

1.15 

1.09 

1.42 

1.17 

1.40 

1.14 

Median 

1.20 

1.06 

1.17 

1.17 

1.20 

1.14 

C.  LESSONS  LEARNED  FROM 
MISSILE  PROGRAMS 

This  section  highlights  the  reasons  for  cost 
and  schedule  growth  in  selected  tactical 
missile  programs  based  on  case  studies  from 
[10],  The  systems  discussed  here  were 
chosen  on  the  basis  of  information 
availability  and  do  not  represent  a  scientific 
sample.  They  do,  however,  include  instances 
of  programs  with  very  high  and  very  low 
cost  and  schedule  growth. 


a.  Reasons  for  Schedule 
Growth  in  Development 

Table  3  summarizes  the  key  program 
characteristics  that  led  to  low  or  high 
schedule  growth  in  development. 
Development  is  highlighted  here,  because  it 
clearly  represents  a  program  outcome~it  is 
important  that  the  system  be  finished  on 
time.  Schedule  growth  in  production,  by 
contrast,  is  often  a  policy  variable— e.g., 
Congress  decides  to  slow  the  program  for 
budgetary  reasons. 
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Table  3.  Characteristics  of  Missile  Programs:  Schedule  Growth  in  Development 
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Two  of  the  three  programs  with  low 
development  schedule  growth — the  TOW  2 
and  the  Sidewinder  AIM-9M — were 
modification  programs  with  fairly  simple 
technologies.  The  other  program,  the  MLRS, 
was  urgent  and  made  a  technical  tradeoff  to 
meet  the  schedule.  Programs  with  high 
development  schedule  growth  included  two 
joint-service  programs  (AIM-9L  and  AIM- 
7F)  and  two  programs  with  schedule  urgency 
(AGM-65D/G  and  AIM-9L). 

Underestimation  of  technical  difficulty  was 
common.  In  two  cases  (AIM-54A  and  AIM- 
91.,),  development  was  slowed  to  resolve 
technical  problems,  in  the  hope  that  they 
would  not  spill  over  into  production.  In  the 
case  of  the  Phoenix  AIM-54A,  that  strategy 
appeared  to  be  successful.  In  the  case  of  the 
Sidewinder  AIM-9L,  it  cannot  be  definitely 
stated  that  the  high  production  cost  growth 
was  due  to  development  problems  spilling 
over  into  production  because  the  AIM-9L 
also  suffered  a  major  production  stretchout. 

b.  Reasons  for  Cost  Growth 

(Total  Program) 

Table  4  summarizes  the  key  program 
characteristics  that  led  to  cost  growth 
outcomes,  based  on  the  case  analyses.  As  in 
the  previous  analysis,  programs  are  grouped 
into  low  and  high  categories. 

All  four  programs  with  the  lowest  TPCG  are 
characterized  by  low  stretch  in  production. 
Interestingly,  three  of  the  four  programs  in 
this  group  were  also  characterized  by 
urgency.  In  the  case  of  the  MLRS,  the 
requirement  was  modified  to  meet  a 
deadline.  (The  AIM-9M  may  be  something 
of  an  anomaly  in  this  group,  because  it  was 
intended  to  fulfill  the  technical  goals  for  the 
AIM-9L.) 


The  programs  with  high  TPCG,  by  contrast, 
were  characterized  by  stretched  production 
schedules.  Both  AMRAAM  and  the  Phoenix 
AIM-54C  had  high  levels  of  concurrency  and 
rushed  testing  programs.  Both  the  Phoenix 
AIM-54C  and  the  Sidewinder  AIM-9L  were 
dual-sourced  for  technical  reasons  (e.g.,  to 
get  a  better  functioning  system)  rather  than 
principally  for  cost  savings.  In  the  case  of  the 
A1M-54C,  funding  was  reduced  to  move 
more  quickly  to  the  next-generation  system, 
and  five  years  were  spent  qualifying  a  second 
source  for  only  two  years  of  head-to-head 
competition,  with  resulting  inefficiencies. 
AMRAAM  also  had  used  a  dual-sourcing 
strategy  and  was  produced  at  less  than  the 
planned  rate— an  expensive  combination.  The 
Sparrow  AIM-7M  also  suffered  from 
reduced  production  funding  in  order  to  fund 
the  next-generation  system. 

D.  LESSONS  LEARNED  FROM 

TACTICAL  AIRCRAFT 

PROGRAMS 

a.  Development 

Tactical  aircraft  outcomes  in  both 
development  and  production  exhibited  less 
variability  than  the  outcomes  of  tactical 
missiles.  Development  cost  growth  ranged 
from  5  to  53  percent,  and  total  program  cost 
growth  ranged  from  -9  to  40  percent.  In 
part,  this  may  be  because  the  sample  of 
aircraft  is  smaller  than  the  sample  of  missiles. 
However,  it  is  also  the  case  that  development 
programs  of  aircraft  appear  to  proceed  more 
smoothly  than  those  of  other  types  of 
systems  [2  and  5], 
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Development  schedule  growth  was  small  in 
the  aircraft  development  programs.  The 
highest  value  was  18  percent  for  the  F-14A. 

The  two  programs  with  the  lowest 
development  cost  growth  are  the  F-5E  and 
the  F-15A/B.  The  F-5E  was  a  relatively 
simple  development  program,  building  on  a 
commercial  system.  The  F-15  program 
proceeded  smoothly,  with  relatively  few 
engineering  changes.  The  F-14A  had  the 
highest  development  cost  growth,  53 
percent.  This  may  have  been  due  to  the 
unusual  contracting  arrangement,  a  fixed- 
price  development  contract  with  options  for 
several  years  of  production.  Grumman  ran 
into  cost  trouble  and  insisted  on  changes  in 
the  contract  before  it  would  produce  the 
aircraft. 

b.  Production 

Few  studies  have  analyzed  aircraft 
production.  One  reason  may  be  that  "the 
production  phase  is  assumed  to  be  less 
interesting  as  outcomes  at  this  point  are 
more  predictable  than  during  development, 
and  the  problems  that  do  occur  are  generally 
traceable  to  faulty  decisions  made  earlier  in 
the  program"  [6,  p.  159], 

But  from  the  contractor's  viewpoint, 
production  opportunities  define  the  business 
environment  contractors  face.  A  contractor 
cannot  keep  going  on  developments  alone. 
According  to  Mayer,  "without  existing  or 
potential  production  work,  contractors  scale 
back  investment,  reduce  staffing,  and 
redirect  efforts  toward  other  markets  that 
have  a  brighter  outlook." 

As  compared  with  the  1950s,  the  1990s  have 
fewer  new  production  starts,  fewer 
production  lines  open  at  any  given  time,  and 
longer  production  runs,  but  with  smaller 
total  quantity— i.e.,  a  lower  production  rate. 


None  of  these  trends,  of  course,  is  at  all 
favorable  to  reducing  costs. 

During  the  1960s,  there  was  a  trend  toward 
fewer  new  production  starts.  However,  the 
Vietnam  War  greatly  increased  the  demand 
for  aircraft,  and  production  buys  were  quite 
large.  The  space  program  also  helped  to 
keep  those  contractors  who  were  frozen  out 
of  the  new  airframe  business  going.  In  the 
1970s,  several  new  programs  started,  but 
annual  buys  were  reduced  after  the  war 
ended.  In  the  1980s,  many  new  programs 
started,  but  not  all  of  them  survived.  The 
1990s  will  be  a  decade  where  only  a  few 
prime  contractors  for  tactical  aircraft  will 
survive. 

The  A-X  program  has  already  been  canceled, 
and  it  appears  that  the  F-22  and  the  F/A- 
18E/F  may  be  the  only  tactical  aircraft 
programs  in  production  by  the  end  of  the 
decade.  Several  systems  currently  in 
production— the  F-14,  the  F-15,  and  the  F- 
16— are  expected  to  end  within  the  next  few 
years. 

Nevertheless,  one  characteristic  of  the 
production  environment  is  that  production 
lines  tend  to  stay  active  much  longer,  even  as 
the  total  number  of  units  produced  declines. 
Table  5  shows  the  average  length  of 
production  runs  for  fighter  and  attack 
programs.  Continuing  these  trends  and 
combining  it  with  a  trend  toward  fewer  new 
programs  provides  further  weight  to  the 
conclusion  that  fewer  units  will  be  produced 
per  year  and  that  contractors  who  fail  to 
participate  in  one  of  the  few  new  programs 
will  no  longer  be  prime  contractors.  A  model 
cited  in  Reference  [6]  projected  that  a 
company  would  lose  its  viability  as  a  prime 
contractor  after  only  18  months  without  a 
contract,  after  which  it  would  be  forced  to 
focus  on  subcontracting. 
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Table  5.  Average  Length  of  Production  Runs  for 
Air  Force  and  Navy  Fighters 


Years 

Length  (Y ears) 

1951-1960 

9.4a 

1961-1970 

20.0 

1971-1980 

18.0 

Source:  Reference  [6]. 

a  Excludes  the  F-5  fighter,  which  was  built 

primarily  for  export. 

In  the  sample  of  tactical  aircraft,  there  was 
considerable  variation  in  production  schedule 
stretch.  The  F-5E  program,  for  example, 
produced  its  planned  quantity  30  percent 
faster  than  planned,  while  the  F-14A  and 
AV-8B  programs  were  stretched  out, 
producing  at  about  half  the  planned  rate. 

With  respect  to  cost,  the  F-5E  came  in  21 
percent  under  expected  cost,  benefiting  from 
a  high  production  rate  and  technical 
simplicity.  However,  both  the  AV-8B  and 
the  F-14A  suffered  from  production  stretch, 
yet  exhibited  low  production  cost  growth.  In 
the  case  of  the  F-14A,  slower  production 
was  accompanied  by  funding  cuts.  The 
original  plan,  to  produce  463  aircraft  in  5.5 
years,  was  recognized  early  as  unrealistic, 
and  the  contractor  had  plenty  of  time  to  scale 
back  production  facilities.  Moreover,  the 
development  of  the  F-14D  aircraft  occurred 
at  about  the  same  time,  and  may  have  helped 
the  contractor  cope  with  costs  on  the  F-14A. 
In  the  case  of  the  AV-8B,  the  program  never 
produced  as  many  aircraft  as  planned. 
However,  the  presence  of  other,  larger 
programs  in  the  same  plant— including  the 
F/A-18  and  the  F- 15— may  have  helped  to 
contain  costs. 

The  program  with  the  highest  production 
cost  growth  is  the  F/A-18,  which  exhibited 
production  cost  growth  of  42  percent, 


despite  little  production  stretch.  The  F/A-18 
originated  as  the  losing  entry  in  a 
competition  to  provide  a  lightweight  fighter 
for  the  Air  Force  and  the  Navy.  The  Air 
Force  selected  the  YF-16  in  a  prototype 
flyoff  over  the  YF-17.  Congress  encouraged 
the  Navy  to  adapt  the  YF-16  to  its  special 
needs.  However,  the  Navy  redesignated  the 
YF-17  as  the  F-18.  Technical  problems  in 
range,  cycle  time,  strafing,  roll,  and  faulty 
radar  images  were  identified  at  the  DSARC 
III  meeting.  Despite  claims  that  the  digital 
flight  control  system  would  eliminate  the 
need  to  physically  change  the  test  aircraft 
(because  changes  would  be  made  in  the 
software),  the  Navy  and  the  contractor  made 
more  physical  changes  to  the  F-18  than  to 
any  other  fighter  aircraft  in  the  last  twenty 
years  [9],  Technical  changes,  particularly  late 
in  development  or  in  production,  are 
expensive. 

D.  CONCLUSION 

This  paper  discusses  research  to  describe 
cost  and  schedule  growth  patterns  associated 
with  the  acquisition  of  major  systems,  to 
identify  reasons  for  the  growth,  and  begins 
to  develop  a  way  to  anticipate  likely  growth 
in  development  and  early  production  phases. 
By  looking  at  past  acquisition  programs,  the 
management  policies  and  circumstances  that 
separate  the  kinds  of  programs  completed  on 
schedule  and  within  cost  plans  from  those 
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that  experience  cost  and  schedule  growth 
can  be  determined. 

Cost  and  schedule  growth  for  the  tactical 
missiles  in  the  sample  were  measured  in 
development  and  in  production.  These 
measures  showed  a  great  deal  of  variability 
among  the  twenty  programs  examined. 
Programs  took  from  50  months  to  134 
months  from  Milestone  II  to  initial 
operational  capability.  Only  one  of  the 
tactical  missile  programs  finished 
development  on  time.  The  program  with  the 
highest  development  schedule  growth 
exceeded  its  plan  by  180  percent.  Two 
programs  were  completed  under  budget, 
while  two  others  cost  more  than  double  their 
Milestone  II  plan. 

The  histories  of  the  tactical  missile  programs 
were  examined  to  determine  the  reasons  for 
schedule  and  cost  growth.  Keys  to 
preventing  schedule  growth  in  development 
are  technical  realism  and  willingness  to  make 
tradeoffs.  Programs  with  high  development 
schedule  growth  tended  to  underestimate 
technical  difficulty.  Two  of  the  five  programs 
with  high  DSG  also  had  high  overall  cost 
growth.  However,  in  three  of  the  five  cases 
of  high  DSG,  it  appeared  that  a  strictly 
phased  approach— resolving  problems  in 
development  when  spending  levels  are  low- 
resulted  in  lower  levels  of  overall  cost 
growth.  Keys  to  preventing  overall  cost 
growth  are  correctly  estimating  the  degree  of 
technical  difficulty  in  the  programs  and 
maintaining  the  planned  production  schedule. 
Programs  that  employed  a  high  degree  of 
concurrency,  that  had  to  be  dual-sourced  for 
technical  reasons  or  that  were  dual-sourced 
at  less  than  full  rate  had  high  cost  growth.  In 
one  case,  the  threat  of  competition  appeared 
to  reduce  costs. 


Cost  and  schedule  growth  measures  were 
also  calculated  for  a  sample  of  seven  tactical 
aircraft.  These  measures  are  less  dispersed 
than  those  for  tactical  missiles.  The  aircraft 
programs  tended  to  receive  more 
management  attention  and  more  protection 
from  schedule  stretch  than  the  tactical 
missiles.  The  highest  cost  growth  index  for 
tactical  aircraft  was  1.40,  versus  2.23  for  the 
tactical  missiles. 

In  two  cases  (the  F-14A  and  the  AV-8B), 
programs  were  stretched  in  production  but 
did  not  suffer  from  the  high  cost  growth  seen 
in  the  missile  programs.  In  the  case  of  the  F- 
14 A,  this  could  have  been  due  to  a 
combination  of  early  warning  and  the 
presence  of  the  F-14D  development  program 
to  cushion  the  blow.  When  the  AV-8B  was 
stretched  out,  the  fact  that  the  F/A-18  and  F- 
1 5  programs  were  using  the  same  plant  may 
have  helped  to  spread  overhead  costs  and 
contain  cost  growth  in  the  AV-8B  program. 
The  program  with  the  highest  production 
cost  growth  is  the  F/A-18,  which  exhibited 
production  cost  growth  of  42  percent. 
Technical  changes  made  late  in  the  process 
contributed  to  its  high  cost  growth. 

E.  IMPLICATIONS  FOR  DOD 

In  the  light  of  these  results,  DoD  and  other 
personnel  who  review  acquisition  programs 
would  benefit  from  a  review  method  based 
on  detailed  information  about  the  strategies 
and  outcomes  of  past  programs.  The  lessons 
of  history  can  be  used  to  identify  which 
programs  require  more  detailed  examination. 
The  case  analyses  indicate  that  there  are 
many  useful  lessons  to  be  gleaned  from 
historical  perspective,  lessons  that  cannot 
always  be  captured  in  a  quantitative 
estimating  relationship.  For  the  tactical 
aircraft,  the  1990s  will  be  a  decade  where 
only  a  few  prime  contractors  will  survive.  If 
programs  are  stretched  out,  contractors  are 
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not  likely  to  be  able  to  cushion  the  blow  by 
working  on  other  programs. 

At  the  beginning  of  a  weapon  system 
development  program,  the  feeling  prevails 
that  that  program  will  avoid  repeating  the 
problems  of  previous  programs.  Yet  cost  and 
schedule  growth  persist.  Despite  individual 


differences  in  programs,  the  importance  of 
understanding  the  level  of  technical  difficulty 
when  original  schedule  and  cost  estimates 
are  made,  of  strict  phasing  and  vigorous 
testing,  and  of  adhering  to  production  plans 
are  borne  out  by  analysis  of  past  strategies 
and  outcomes. 
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PRODUCT  OR  PROCESS? 


by  Dr.  Franz  A.  P.  Frisch 
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ABSTRACT 

•  Present  discussions  about  the  Industrial 
Base  deal  with  the  PRODUCTS  of  the  base, 
regardless  of  its  military  or  commercial 
nature.  The  tally  of  products  in  a  highly 
developed  economy  might  go  into  the 
millions,  well  beyond  anybody’s  singular 
comprehension. 

•  We  overlook  the  fact  that  large  numbers 
of  products  are  the  result  of  the  relatively 
few  PROCESSES  used  to  make  them. 

•  It  is  suggested  we  develop  a  taxonomy  of 
processes  and  determine  those  of  national 
importance  for  the  future  position  of  our 
nation  and  concentrate  upon  the  preservation 
of  those  processes. 

The  OBJECTIVE  of  this  paper  is 
first  “to  justify”  and  second  “to  outline”  the 
development  of  a  process  taxonomy  and  in 
turn  develop  a  new  vista  from  which  to  view 
present  problems  like  conversion  or  dual- 
use. 

The  SCOPE  of  the  paper  is  limited  to 
the  fundamental  engineering  aspects  of  the 
process  taxonomy.  Political-economic 
aspects  are  only  mentioned  occasionally  in 
order  to  underline  their  importance  and  need 
for  further  studies. 


The  PURPOSE  of  this  paper  is  to 
suggest  a  new  approach  to  problems  of  the 
Industrial  Base  and  to  inspire  a  dialog  on  the 
subject.  The  paper  is  not  intended  to 
provide  a  cookbook  of  how  to  do  things,  but 
rather  an  invitation  to  think  differently  about 
our  challenges. 

The  B  ACKGROUND  for  the  paper 
is  provided  largely  by  the  changed  military- 
political  situation  as  result  of  ending  the  Cold 
War.  Prior  to  this  time,  for  example  in 
Global  War  Game  exercises,  the  main 
question  was,  “How  fast  can  we  increase  our 
military  capabilities  in  case  of  war  or  in  the 
case  of  major  materiel  losses  due  to  an 
enemy  attack?”  But,  with  the  “sea  change” 
of  Communism’s  decline  this  concern  has  all 
but  evaporated. 

The  first  question  in  the  euphoria 
after  the  end  of  the  Cold  War  was  about  the 
proper  utilization  of  the  one-time  defense 
industrial  resources  for  a  popularly  labeled 
“peace-dividend”:  How  can  we  convert  our 
defense  industry  to  a  beneficial  peacetime 
production,  and  well  meaning  people  were 
eager  to  suppose  the  experiences  after  World 
War  II  as  a  model.  But,  soon  it  was 
recognized  that  the  WWII  experience  was 
utterly  meaningless.  After  WWII,  a 
tremendous  pent-up  demand  existed  and 
most  of  the  returning  soldiers  had  real 
savings  to  satisfy  this  demand.  In  contrast, 
after  the  Cold  War,  not  only  was  there  no 
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pent-up  demand  ,  but  a  considerable  over¬ 
saturation  of  the  market.  Differently 
expressed,  no  demand  for  products,  resulting 
from  conversion,  existed  and  the  economic 
situation  now  and  then  is  not  comparable. 

We  at  DSMC,  aware  of  that  truth, 
initiated  a  pilot  study  with  the  Naval 
Shipyard  in  Philadelphia.  Our  question  was, 
“What  goods  can  you  produce  with  your 
existing  equipment,  which  is  presently 
imported  into  the  United  States,  and  hence, 
without  making  competition  to  existing 
American  producers  (working  already  with 
low  facility  utilization)?”  The  answer  was: 
“We  could  produce  all  railroad  cars  presently 
imported  from  Europe  and  most  of  the 
fasteners  imported  from  Taiwan  and 
therefore  not  produced  in  the  United  States.” 

This  pilot  study  was  simply  learning 
tool  for  us.  Unfortunately,  we  had  no 
opportunity  to  extend  this  study  beyond  the 
engineering  aspects  of  so-called  conversion. 
To  study  the  economic  aspect  would  have 
been  even  more  interesting.  For  example, 
one  could  have  asked,  “If  the  U.S.  yard 
would  have  built  railroad  cars  instead  of 
closing,  therefore  further  diminishing  the 
U.S.  industrial  base,  how  would  you  account 
for  for  the  past  investment?”  “Should  it  be 
considered  at  its  book  value  or  as  sunk-cost 
...  etcetera?” 

Slowly,  the  conversion  mania 
subsided  the  important  questions  were 
excavated:  for  example,  under  the  heading 
of  “dual-use”  and  of  “military  industrial 
capability,”  or  “reconstituting”  in  case  of 
need,  combined  with  the  desire  to  stay  in  the 
forefront  of  military-technological 
capabilities  and  R&D  under  the  heading  of 
“research  without  production.” 


The  Office  of  Technology 
Assessment  of  the  Congress  of  the  United 
States  responded  without  delay  to  the  new 
challenges  and  provided  a  series  of  excellent 
publications,  like  “Redesigning  Defense  - 
1991,”  or  “Defense  Conversion  -  1993.”  All 
those  publications  deal  with  the  contours  of 
high  level  manufacturing  policy  for  the 
products  needed  to  defend. 

A  similar  (but  commercially  oriented) 
view  is  taken  by  the  Department  of 
Commerce  (DOC).  In  DOC  studies,  the 
products  are  not  as  dominant  as  are  the 
manufacturing  entities  which  produce  the 
products.  The  Standard  Industrial 
Classification  (SIC)  codes  lists 
approximately  11,000  manufacturing  entitites 
and  the  alphabetic  index  reveals  close  to 
20,000  products  (or  product  groups). 

The  concern  with  products  is  more 
than  natural.  It  is  self  evident.  Irrespective 
of  the  commercial  or  military  domain, 
regardless  if  seen  from  the  government  point 
of  view  or  from  the  point  of  a  private  person, 
not  accounting  for  our  position  as  producer 
or  consumer  ...  we  always  concern  ourselves 
with  products  in  the  form  of  goods  and 
services.  We  want  to  know  the  quality  of 
the  product  and  its  price.  We  are  talking 
about  the  cost  effectiveness  (value)  of 
different  products,  of  international 
competitiveness  and  so  forth,  ad  infinitum. 

The  number  of  products  major 
department  stores  have  to  offer  goes  into  the 
millions.  We  can  live  with  this  and  do  take 
it  correctly  as  a  symbol  for  a  high  quality, 
developed  economy  and  most  of  us  are  really 
not  concerned  “how”  all  those  products  are 
provided. 
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And  here,  the  problem  begins  to 

fester. 

In  a  free  (laissez-faire)  economy,  the 
market  decides  (by  buying  or  not)  what  the 
customer  wants  and  the  system  works  to 
relative  perfection,  without  any  government 
interference.  As  long  as  the  demand  for 
military  goods  (during  war  time  or  the  cold 
war)  was  very  large,  even  the  defense  market 
worked  on  a  quasi-competitive  basis,  where 
supply  responded  to  demand  in  the 
oligopsony  (a  system  with  one  or  a  few 
buyers  and  many  competing  suppliers).  But 
today  those  conditions  belong  to  the  past. 
Presently,  the  demand  for  military  goods  is 
frequently  below  the  economic  ordering  - 
quantity  (which  can  be  thought  of  as  the 
minimum  sustainable  production  rate)  and 
hence  many  defense  suppliers  are  forced  out 
of  the  market.  The  Department  of  Defense 
reacted  to  this  new  status  by  shifting  from 
“the  best  buy”  paradigm  to  the  more  basic 
“maintenance  of  certain  capabilities.”  For 
example,  the  Navy  is  concerned  to  keep  one 
builder  of  submarines  (Groton  Electric  Boat) 
and  one  builder  of  aircraft  carriers  (Newport 
News)  relatively  solvent.  The  Air  Force 
Assistant  Secretary  for  Acquisition,  Mr. 
Clark  Fiester  stated  (Airman  magazine, 
February  1995)  that  “We  can  no  longer 
afford  a  separate  military  industrial  base”  ... 
“while  producing  superior  weapons  .” 

Nothing  could  describe  the  changed 
situation  better  than  Fiester’ s  words.  It 
simply  describes  reality.  The  problems  are 
recognized.  But,  what  should  be  the  action- 
response? 

The  “possible”  action  responses  will 
be  bounded  or  be  located  somewhere 
between  two  extremes.  The  first  extreme  is 
to  subscribe  completely  to  the  laissez-  faire 
philosophy  and  assume  whatever  the  military 


ordering  quantities  will  be  in  the  future,  the 
supply  side  of  the  market  will  respond  to  it 
and  government  interference  with  the  private 
industry  will  be  neither  necessary  nor 
appropriate.  The  second  extreme  would  be 
to  subscribe  to  a  full  control  of  the  supply 
side  for  military  goods  through  the  (re) 
introduction  of  a  arsenal  system  (which  is 
tantamount  to  nationalization  of  weapons 
production),  at  least  for  those  weapon 
systems  which  are  considered  critical  to  the 
military  posture  of  the  United  States.  But, 
since  extremes  are  mostly  wrong  and  hardly 
ever  optimum,  a  search  for  a  middle  position 
is  necessitated.  The  search  for  this  middle 
position  between  the  two  extremes  has  two 
decision  engines:  Judgment  and  Knowledge, 
(two  terms  which  need  at  least  some 
explanation). 

Every  decision  process  has  three 
phases.  At  the  beginning  and  the  end  is 
judgment  and  in  between  analysis  based  on 
knowledge.  Judgment  at  the  first  inkling  is 
mostly  quite  of  an  abstract  nature.  It  is 
governed  by  a  tapestry  of  values,  with  a 
singular  view  toward  the  future,  with  our 
desires  and  wishes  like  “we  have  to  stay  in 
the  forefront  of  technology”  or  “we  have  to 
be  on  the  leading  edge.” 

“Judgment  -  at  the  beginning”  is  the 
goal  setter  in  national  and  private  life.  In 
short,  the  initial  judgment  expresses  the  “I 
WANT.” 

After  the  “I  want”  is  defined,  follows 
the  analytical  fora:  What  do  I  have  to  know 
in  order  to  reach  the  goal  described  with  the 
“I  want?”  What  resources  do  I  need?  How 
much  will  it  cost  to  reach  the  goal? 

The  analytical  part  is  eminently 
visceral  yet  concrete.  Everything  is 
measurable  and  countable:  How  is  the 
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industrial  base  structured?  What  are  the 
value-added  processes?  What  capital/labor 
ratios  are  involved? 

After  the  analytical  part  is 
culminated,  we  return  to  the  “judgment  at 
the  end:”  After  the  analysis  has  determined 
what  “I  CAN  DO”  and  then  its  price  tag 
becomes  the  question:  “Am  I  willing  to  pay 
the  price?”  This  is  pure  judgment  (values), 
unsupported  by  the  middle  phase  logic. 

Judgment  -  in  its  most  generalized 
form  -  will  determine  the  future  role  of  the 
United  States  defense  establishment  and  an 
estimate  about  the  time  we  will  have  to 
reconstitute  the  capability  to  produce 
military  goods  in  the  case  of  an  emergency 
(if  such  capabilities  should  be  lost).  This 
judgment,  like  in  the  past,  could  be  translated 
into  “requirements.” 

However,  I  doubt  that  this  judgment 
can  be  realized  without  the  knowledge  about 
the  industrial  base  (of  this  what  CAN  be 
done)  ...  and  furthermore,  I  doubt  that  this 
knowledge  can  be  acquired  as  long  as  we  are 
talking  about  “products”  regardless  of  their 
commercial  or  military  value.  We  simply 
have  too  many  products  to  ever  be  fully 
comprehended  by  analysts  or  the  collector  of 
knowledge. 

The  conventional  wisdom  in  case  of 
an  over  abundance  of  “data”  would  be  to 
revert  to  statistical  sampling  as  a  supposed 
representive  reality.  No  question,  this  could 
help  and  the  wide  collection  of  statistical 
data  by  the  Department  of  Commerce  could 
provide  the  starting  point. 

However,  I  think  we  could  solve  the 
problem  of  knowledge  accumulation  much 
easier  by  shifting  from  the  products  to  a 
consideration  of  the  relative  few  processes 


[which  in  turn  are]  leading  to  the 
incomprehensible  abundance  of  products. 

For  example,  the  knowledge  of  the 
value  added  process,  of  the  capital/labor 
ratio,  etcetera,  is  NOT  product  dependent, 
but  process  dependent  and  therefore  valid  for 
all  products  subject  to  the  same  process. 

History  of  manufacturing  supports 
my  suggestion  for  the  shift  of  our  attention 
from  products  to  processes.  Let  me  give 
you  three  examples: 

(1)  In  the  long  bygone  days,  for 
example,  a  carpenter  or  furniture  maker  was 
the  master  of  four  processes:  Of  cutting 
wood;  of  planing  wood;  of  connecting  wood, 
and  of  finishing  wood.  With  these  four 
processes,  he  was  able  to  build  a  wide 
variety  of  furniture  according  to  the 
individual  choice  of  his  customers.  He  might 
never  have  built  the  same  piece  twice. 

(2)  Many  leading  car  manufacturers 

in  Europe  between  1910  and  1930  were  a 
collection  of  masters  of  the  different  metal- 
trades:  casting;  forging;  metal  forming; 

leather  work,  etcetera  and  with  a 
combination  of  those  activities  were  able  to 
manufacture  luxury  cars,  buses,  trucks, 
trailers,  taxi  cabs.  Whatever  was  running  on 
wheels;  in  small  quantities.  It  was  not  very 
efficient,  but  the  mastery  of  a  few  processes 
enabled  them  to  respond  to  a  wide  range  of 
demands. 

(3)  After  WWXI,  when  the  European 
ship  building  industry  started  to  recuperate, 
almost  all  shipyards  had  to  be  “integrated 
yards,”  able  to  make  anything  from  the  hull 
to  the  engine,  pumps,  propellers,  cranes  and 
so  forth,  within  their  own  boundaries.  The 
lack  of  a  supply  industry  forced  them  to  do 
this.  They  were  able  to  do  it,  because  they 
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had  mastered  all  processes  of  the  metal 
trades.  At  this  time,  the  United  States  ship 
building  industry  could  already  afford  to 
work  with  “assembly-yards,”  supported  by  a 
highly  developed  network  of  subcontractors, 
specializing  in  engines,  pumps,  cranes, 
propellers  and  so  forth. 

I  hope  those  few  examples 

demonstrate: 

•  As  long  as  we  remain  masters  of 
relatively  few  processes,  we  will  be  able  to 
build  a  wide  variety  of  products.  For 
example,  as  long  as  we  are  able  to  preserve 
the  knowledge  embedded  in  workers  and  the 
facilities  to  weld  high-tensile  steel,  we  will  be 
able  to  build  tanks,  submarines,  landing 
gears,  missiles  and  everything  else  for  the 
military  and  commercial  customer  —  using 
mastery  of  the  rudimentary  processes  as 
opposed  to  the  more  shallow  application  of 
pre-fabricated  products. 

•  When  we  lose  specific  process 
capabilities,  nurtured  in  workers  using  some 
general  tools,  we  have  little  chance  to 
reconstitute  whenever  needed. 

I  hope  my  few  examples  and  logic  for 
the  concentration  on  processes  instead  of 
products  should  suffice  to  wake  the  readers 
interest  toward  my  suggestions. 

Now  comes  the  difficult  part:  The 
ANALYSIS  of  my  suggestions.  I  will  try  to 
outline  the  analysis  in  a  few  distinct  steps: 


•  PRODUCT  LEVELS 

•  PROCESS  TAXONOMY 

•  DEFINITIONS 

•  RESTRUCTURING  and 

•  DATABASE 


Product  Levels 

Everything  which  deserves  the  name 
“system,”  like  a  ship,  an  aircraft  or  a  tank,  a 
car  or  a  TV  set  can  be  dissolved  into  a  work 
breakdown  structure  (WBS)  as  shown  in 
Table  I. 

The  product  levels  shown  in  Table  I 
are  valid  for  any  product  whatsoever  and 
provide  the  first  indication  of  the  relevant 
processes  under  the  heading  of  “key  activity” 
at  each  level: 

•  Assembling  (at  level  I,  II  and 
III) 

•  Making  (at  level  IV) 

•  Refining  and/or  forming  (at 
level  V),  and 

•  Extracting  (at  level  VI). 

If  we  equate  the  term  of  “key 
activity”  with  “group  of  processes,”  then  we 
have  already  the  first  entry  into  a  process 
taxonomy.  The  four  groups  of  processes  (in 
reverse  order)  beginning  with  Extracting  to 
Refining  to  Making  and  finally  to 
Assembling,  indicates  also  the  “value-added 
structure”  of  every  product.  Value-added 
(VA)  describes  the  capital  plus  labor  applied 
at  each  level.  The  output  (P)  at  each  level 
enters  the  next  in  the  form  of  material  (M). 
In  this  hierarchy,  we  have  as  input  the  raw 
material  (RM),  six  value-added  inputs  (VA) 
and  one  single  output  (Pi)  as  shown  in  figure 
1. 

Each  value-added  entry  describes  a 
specific  process  at  each  particular  product 
level  in  its  most  fundamental  form  and  this 
value-added  chain  is  ubiquitous  for  all  final 
systems,  regardless  if  we  are  concerned  with 
military  or  commercial  products  and 
regardless  of  the  multitudes  of  the  possible 
industrial  structures  and/or  possible 
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ownership  variations,  ranging  from  individual 
ownership  at  each  level  to  the  complete 
vertical  cartel  structure.  The  Ford  Motor 
company,  for  example,  once  owned 
everything  from  an  ore  mine  to  a  rubber 
plantation  in  Brazil  to  the  assembly  line  for 
its  cars.  The  organizational  and  ownership 
structures  are  sketched  (in  sample  form)  in 
figure  1. 

The  organizational  structure  can  be 
selected  according  to  the  economic  facts  at  a 
given  time.  From  the  eight  options,  only 
three  are  shown.  Common  to  all  options  is 
only  the  step  from  the  level-VI  to  level-V. 
From  there,  the  material  can  directly  be 
delivered  to  level-I  (path  P-1)  or  to  any 
intermediate  level  (path  P-2  and  P-3). 
Economy  of  scope  and  economy  of  scale  will 
drive  this  organization,  and  the  related 
decision  is  frequently  referred  to  as  the 
make-or-buy  decision. 

The  ownership  structure  is  the  result 
of  the  combination  of  the  organizational 
structure  plus  the  freedom  of  association  as 
given  by  the  different  corporate  laws  of  the 
nations.  Antitrust  laws,  laws  of  competition 
fall  into  this  category.  A  total  of  32 
collocated  ownership  structures  are  possible, 
but  only  3  are  shown.  Non  collocated 
structures  where,  for  example,  the  steel  mill 
(level-V)  owns  the  shipyard  (level-I)  are  not 
considered: 

Process  Taxonomy 

It  should  be  relatively  easy  to  break 
down  existing  systems  like  air  crafts,  ships, 
tanks,  missiles  and  so  forth,  into  the  related 
product  levels.  However,  in  order  to 
develop  a  process  taxonomy,  we  have  to 
differentiate  each  product  at  each  level,  but 
at  least  for  the  levels  II  through  VI  the 
nature  of  the  product,  such  as 


•  Structural  Products 

•  Mechanical  Products 

•  Electrical  Products 

•  Electronic  Products,  and 

•  Chemical  Products. 

These  five  “natures  of  products”  can 
form  together  with  the  previous  four  “group 
of  processes”  a  NATURE-PROCESS 
MATRIX.  At  each  node  of  this  matrix,  most 
specific  processes  can  be  entered  such  as 
metal  forming,  metal  connecting,  metal 
protecting  and  so  forth.  This  would  provide 
us  with  the  first  step  in  the  development  of  a 
process  taxonomy. 

The  second  step  must  be  the 
development  of  PROCESS  PARAMETERS. 
Some  of  the  most  important  process 
parameters  might  be 

•  Quality 

•  Weight,  and 

•  Quantity. 

Quality,  for  example,  for  mechanical 
parts  or  elements  (level  IV)  can  be  described 
by  the  required  tolerances  or  precision 
needed  for  the  manufacturing  process.  The 
quality  determines  the  type  of  equipment  and 
also  the  skill  of  the  operator  as  needed  for  a 
specific  manufacturing  process. 

Weight  of  the  elements,  components 
and  subsystems  (level  II,  III  and  IV) 
determines  the  lift,  or  crane  capacity  as 
needed  to  transport  them  within  the  factory 
or  between  factories. 

Quantity,  finally,  most  applicable  to 
material  (level  V)  and  chemical  products 
determines  the  total  amount  (minimum  and 
maximum)  which  can  be  or  must  be 
produced  with  certain  existing  processes. 
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•  The  combination  of  the  process 
parameters  with  the  nature-process  matrix 
will  be  the  basis  for  a  meaningful  process 
taxonomy. 

This  process  taxonomy,  probably 
consisting  of  a  scant  few  hundred  entries, 
should  be  able  to  represent  thousands  and 
thousands  of  products  ...  of  both  commercial 
and  military  values. 

Definitions 

If  we  are  willing  (as  already 
mentioned  before)  to  accept  a  process 
taxonomy  instead  of  a  product  taxonomy  as 
the  fundamental  ordering  system  of  all 
industrial  activities,  than  we  will  be  able  to 
define  our  present  concerns  about  the 
industrial  base  differently  than  presently 
done. 

•  Instead  of  Dual-Use  we  may  talk 
about  DUAL  PROCESSES  and  define  it  as 
such  a  process  which  can  be  used  for 
commercial  and  military  products  without 
(1)  the  need  for  a  different  knowledge,  (2) 
the  need  for  different  tools  and  (3)  with  the 
same  degree  of  tool  utilization.  In  all  those 
cases  we  can  change  products  “WITHOUT” 
conversion. 


•  Instead  of  the  term  Conversion,  we 
may  talk  about  PROCESS 
CHANGES/REALIGNMENT  and  define  it 
as  a  situation  where  the  existing  processes  in 
a  facility  are  either  (1)  insufficient  for  the 
new  product  or  (2)  unnecessary  for  the  new 
product  or  (3)  will  result  in  an  entirely 
different  tool  utilization  than  previously 
needed. 


•  The  term  of  reconstitution  might 
be  replaced  by  PROCESS 
PRESERVATION  for  all  those  processes 
which  are  UNIQUE  for  military  products 
and  have  no  practical  application  for 
commercial  products,  for  example,  the 
welding  of  super-heavy,  high-tensile  metal 
plates  as  needed  for  submarine  construction. 
Processes,  recognized  and  defined  as  unique 
for  militaiy  products,  must  be  either 
artificially  preserved  (at  high  cost)  through 
training  exercises  for  a  selected  work  force 
or,  if  possible,  replaced  through  research  for 
new  (more  robust)  dual-use  processes.  If  we 
do  this,  the  concept  of  “research  without 
production”  may  get  a  new  center  of  interest 
for  research  activities  and  might  also 
strengthen  the  material  research. 

Restructuring 

Restructuring  might  be  the  most 
difficult  problem  to  address.  It  is  the 
problem  of  rebuilding  in  the  future  elements 
of  the  “military  industrial  base,”  whenever 
needed  in  a  changed  political  situation  and 
after  we  have  lost  the  existing  capabilities. 
The  crucial  questions  (according  to  models 
built  for  the  Global  War  Games  of  the  past) 
where  first  how  long  will  it  take  to 
reconstitute  the  base  and  second  to  what 
volume  of  products  can  the  base  be 
reconstituted  to  in  a  given  time  frame. 

I  suggest  consideration  of  the 
application  of  the  proposed  process  concept 
for  the  problem  of  reconstitution.  If  so,  then 
two  base-situations  can  be  imagined: 

•  first,  dual-use  (robust)  processes 
exist,  but  are  at  a  given  time  exclusively  used 
for  commercial  purposes  by  specific 
individual  manufactures  of  commercial 
goods,  and 
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•  second,  the  processes  are  in 
existence,  but  distributed  among  competing 
companies. 

In  the  first  case,  we  will  have  to  find 
a  way  to  shift  the  commercial  production  to 
military  production  and  in  the  second  case, 
we  will  have  to  search  for  a  way  to  re-orient 
competitors  to  collaborators  -  in  both  cases, 
assuming  sufficient  quantitative  capabilities 
exist  to  possibly  satisfy  the  restructured 
demand. 

Unfortunately,  in  both  cases,  the 
commercial  supply  will  have  to  be  reduced  in 
the  interest  of  military  supply  and  the  “guns 
or  butter”  trade-off  may  be  realized.  With 
this  remark,  we  are  entering  the  political  side 
of  the  problem,  not  subject  of  the  present 
paper.  However,  we  shall  not  forget  it. 

Data  Base 

Presently,  extensive  data  collection 
exists  (i.e.,  SIC-code)  about  “products,”  but 
practically  no  data  exists  about  processes. 
This  however  does  not  mean  that  this 
knowledge  does  not  exist.  Every  technical 
university  is  teaching  the  essence  of  products 
AND  the  processes  needed  to  provide  those 
products.  Therefore,  it  can  be  assumed  that 
teachers  on  those  institutions  will  be  able  as 
step  #1  to  identify  generically  the  type  of 
processes  in  the  structural,  mechanical, 
electrical,  electronic  and  chemical  field  of 
activity.  As  step  #2.  it  should  be  possible  to 
subdivide  the  generic  steps  into  specific  steps 
according  to  product  parameters  of  quality, 
weight  and  quantity. 

Steps  1  and  2  could  be  performed 
independently  around  the  purpose  of  the  data 
base  to  be  developed.  But,  thereafter,  we 
will  have  to  make  value  decisions  about  the 
specificity  of  the  data  collection: 


•  Shall  the  new  data  base  serve  only 
for  the  clarification/objectivisation  and 
foremost;  quantification  of  the  technological/ 
engineering  value  of  the  industrial  base,  or 

•  Shall  the  new  data  base  also  serve 
for  the  clarification/objectivization  of  the 
economic-political  aspects  of  the  same. 

This  decision  is  important  in  order  to 
MINIMIZE  the  DATA  COLLECTION 
down  to  the  absolutely  necessary  effort. 
Presently,  I  concern  myself  only  with  the 
engineering  aspects. 

Step  3  would  be  the  development  of 
a  technical  questionnaire,  tailored  to  the  type 
of  processes,  from  structural  to  chemical. 
This  questionnaire  would  ask  for  information 
about  the  parameters  for  specific  facilities, 
the  present  utilization  of  the  process  related 
equipment,  the  capital/labor  ratio  for  those 
processes,  the  years  of  experience  for 
masters,  formen,  and  apprentices. 

However,  the  questionnaire  will  not 
ask  any  question  related  to  corporate 
financial  aspects. 

Step  4.  Data  collection. 

Step  5.  Determination  of  “Military 
unique”  processes. 

SUMMARY  AND  COMMENT 

The  paper  outlined  the  logic  for  the 
proposed  shift  from  a  product  taxonomy  to  a 
process  taxonomy.  This  would  change  the 
entire  information  system  about  industrial 
capabilities. 

It  is  clear  to  me  that  a  process  based 
information  system  about  industrial 
capabilities  is  not  necessarily  simple.  But,  it 
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should  be  simpler  than  the  present  product 
based  systems  and  it  should  definitely 
provide  more  information  about  the 
industrial  capabilities  in  peace  and  war. 

It  is  also  clear  to  me  that  the  task  of 
development,  such  a  new  information  system 
needs  the  team  work  of  many  specialists. 

However,  I  am  convinced  it  can  be 
done  and  the  pay-off  will  be  great. 

Finally:  I  invite  your  comments. 
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TO  PRESERVE  AND  PROTECT  THE  DEFENSE  INDUSTRIAL  BASE 

MarkWerfel 


ABSTRACT 

Current  conditions  effecting  program  decisions, 
authorizations  and  appropriations  create  major 
dilemmas  for  Defense  Industry  leaders  who  are 
responsible  to  shareholders  for  profitability  and 
to  their  highly  skilled  employees  for 
employment.  Similar  dilemmas  are  facing  their 
Government  counterparts.  These  conditions 
strain  customer  relationships,  subvert  short  and 
long  term  planning  required  for  goal 
attainment,  interfere  with  capital  investment 
and  may  impact  their  organizational  viability. 
These  factors  and  the  associated  risk  are 
continuing  to  deplete  the  already  declining 
Defense  Industrial  Base  (DIB).  The  possibility 
now  looms  that  the  DIB  will  lose  its  breadth 
and  depth  at  a  time  when  the  nature  of  our 
military  opponents  and  the  type  of  warfare  to 
be  fought  is  less  certain  than  ever  before. 
Further,  the  expected  decline  of  the  DIB,  the 
defense  technology  base  and  future  inability  to 
produce  necessary  systems  questions  effective 
National  commitment  to  defense  requirements. 

The  author  presents  a  new,  partnership 
approach  to  the  DIB,  which  clarifies 
responsibilities  between  the  parties  and  also 
significantly  improves  its  management. 

Key  Issues 

While  the  Federal  Government  is  explicitly 
responsible  for  national  defense,  there  is  an 
implicit  reliance  on  the  private  sector  for 


maintaining  its  bedrock,  the  DIB;  resulting  in 
conflict  for  Industry  leaders  who  are  explicitly 
responsible  to  their  employer  for  profitability  in 
the  long  and  in  the  short  term.  Within  and 
between  each  sector,  without  clear 
responsibility  and  accountability,  and  without 
clear  Government  commitment;  any  long  term 
planning  that  either  does  is  one-sided  and 
therefore  ineffective.  Outcomes  may  include 
organizational  conflict,  poor  customer 
relationships,  suboptimal  and  erratic  planning 
and  programming,  and  the  possibility  that, 
when  needed,  portions  of  the  DIB  required  to 
support  defense  requirements  will  not  be 
available. 

Resulting  Dysfunction 
Industrial  costs 

It  can  be  argued  that  the  DIB  is  not  managed, 
but  is  a  fortuitous  outcome  of  the  acquisition 
process.  Government  financial  support  for 
fixed  and  semi-variable  "industrial"  costs  (e.g., 
for  plant  and  production  equipment)  is  derived 
from  contracts  for  delivery  of  given  quantities 
of  weapons  systems,  which  vary  annually  due 
to  programmatic  and  budgetary  factors. 
Industrial  cost  expenditures  by  industry  are 
very  significant,  even  though  their  recovery 
from  the  Government  customer  is  uncertain. 
Further,  they  are  not  subject  to  managerial 
control  once  committed  to,  and  can  only  be 
recovered  if  the  anticipated  future  work  is 
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actually  received,  and  at  the  anticipated  annual 
volume. 

Initially,  Industry  may  be  required  to  commit 
capital,  building  capacity  in  anticipation  of 
program/phase  authorization  and  funding, 
simply  to  qualify  as  a  potential  participant.  For 
the  awardee,  industrial  cost  risk  then  increases, 
because  those  costs  may  not  be  fully 
amortized/recovered.  Production  contract 
pricing,  when  performance  extends  beyond  the 
current  year,  1)  assumes  additional  contract 
work  will  be  awarded  in  the  outyears  and  2) 
overhead  costs,  including  industrial  costs,  will 
be  absorbed  by  current  and  estimated  future 
work.  If  DOD  work  levels  do  not  materialize 
or  drop,  outyear  actual  overhead  rates  may  be 
much  higher  than  anticipated,  given  they  will  be 
allocated  across  a  smaller  business  base. 
However,  current  fixed  price  contract  work 
can  not  be  repriced.  This  is  today's  dilemma 
when,  unexpectedly,  production  may  be 
eliminated  or  curtailed. 

The  vulnerability  of  plants  that  are  dedicated  to 
one  or  to  a  very  few  customers  is  particularly 
and  severely  high  in  periods  of  DOD 
drawdown/program  stretch-out.  As  a  result  of 
DOD  funding  reductions,  weapons  systems 
quantities  are  being  reduced.  On  previously 
awarded  contract  work,  while  total  industrial 
costs  would  remain  the  same,  outyear  per-unit 
costs  estimated  during  the  original  contract 
negotiations  would  be  greatly  lower  than  unit 
actual  costs.  If  the  resulting  unit  price  increase 
is  not  accepted,  understood  or  acknowledged 
by  Government  acquisition  officials  and/or  by 
Congress,  it  may  lead  to  additional  unit 
quantity  reductions  or  to  premature  program 


cancellation.  Potential  or  actual  cancellation  of 
a  major  program  that  constitutes  most  or  all  of 
a  given  plant's  business  base  clearly  is  a  key 
concern  for  both  parties;  with  many  obvious 
impediments  to  their  objectives.  Further,  the 
effect  cascades.  In  a  two  or  three  program 
plant,  one  program's  cancellation  will  increase 
the  remaining  programs'  unit  costs,  and  could 
lead  to  their  cancellation  as  well.  Clearly,  the 
situation  is  not  "win-win." 

In  addition  to  industrial  costs,  recovery  of  other 
costs  is  also  uncertain,  due  to  Government 
contract  pricing  policies,  which  frequently 
change,  and  often  strain  the  essential  principle 
of  fair  and  reasonable  pricing.  In  the  past,  fixed 
price  development  contract  prices  reflected  less 
than  full  anticipated  costs;  and  certain  normal 
business  costs,  such  as  for  advertising/public 
relations  or  IR&D,  were  unallowable  or 
allowable  up  to  an  externally  imposed 
threshold;  despite  their  value  to  the  corporation 
and/or  to  national  defense. 

The  need  for  long  term  commitment 

vs.  planning 

The  Government  acquisition  process,  heavily 
influenced  by  political  and  budgetary  forces, 
does  not  foster  the  stable  business  relationship 
that  would  benefit  both  parties.  During  the 
past  decade,  much  has  been  said  about  the  need 
for  long  term  planning  (and,  by  inference, 
commitment  to  plans  once  accepted).  Instead, 
the  situation  has  remained  unstable.  For 
example,  the  "peace  dividend"  is  a  self-fulfilling 
prophecy,  which  may,  in  turn,  reduce  defense 
budgets,  degrade  or  constrain  the  defense 
industrial  base,  and  reduce  its  future  ability  to 


COPYRIGHT  1993,  1995  MARK  WERFEL  ALL  RIGHTS  RESERVED 

Please  note  that  the  views  expressed  in  this  paper  are  those  of  the  author 
and  do  not  necessarily  reflect  the  views  of  the  Department  of  the  Army. 


446 


replicate  a  high  technology  based  "Desert 
Storm"  success.  From  the  time  of  the 
"Carlucci  principles"  to  today's  "Total  Quality 
Management  and  Re-Engineering,"  the 
desirability  of  long  term  planning,  improving 
processes  and  eliminating  unneeded  or 
unwarranted  activities  directed  to  reduce 
significant  variation  in  their  outputs  have  been 
championed,  but  not  achieved.  Instead,  the 
acquisition  process  state  and  such  concomitant 
results  as  the  substantial  price/cost  variation 
described  above  is  inconsistent  with  and 
interferes  with  customer  relationships  (note,  the 
customer  may  mean  the  program  manager, 
Defense  leadership,  the  Congress  or  the 
public). 

National  priorities  are  shifting  and  the  DIB 
must  respond.  The  military  threat  is  being 
reduced,  but  its  nature  is  changing.  Our  focus 
will  shift  to  such  issues  as  the  industrial 
infrastructure  and  the  environment.  While 
large  active  military  forces  and  weapons 
inventories  may  no  longer  be  afforded  or 
required,  the  DIB  structure  must  correlate  with 
any  expected  military  threat,  from  a  Desert 
Storm  to  a  Somalia;  and  do  so  in  such  terms  as 
technology/weapons  mix,  abbreviated 
leadtimes  and  logistics. 

Traditionally,  the  DIB  received  less  attention 
because  defense  products  were  less  complex, 
and  many  could  be  manufactured  by 
commercial  plants,  with  substantial  leadtime 
afforded  by  our  distance  from  potential 
enemies.  This  is  not  true  today.  Sophisticated 
weapons  systems  drive  special  purpose  plant 
and  equipment  (and  the  "industrial  costs" 
discussed  before);  which  render  Defense  plants 


unlikely  or  unable  to  convert  to  commercial 
use.  Further,  once  allowed  to  deteriorate,  not 
only  will  the  DIB  be  less  flexible  for  various 
conflict  scenarios,  require  more  time  to  ramp 
up  but,  in  certain  situations,  may  be  more 
costly  on  a  net  basis  due  to  shut  down/start  up 
costs.  Similarly,  skilled  employees  and 
production  related  experience  may  be 
unavailable  if,  as  currently  planned,  weapons 
system  programs  do  not  progress  past  the  full 
scale  development  phase.  Due  to  the  increase 
in  contractor  financial  risk  apparent  in  the 
drawdown,  industry  may  not  participate  or  may 
demand  much  higher  profit  rates  than 
previously. 

IMIP,  an  earlier  approach 

The  DOD's  Industrial  Modernization  Incentives 
Program  (fiVUP)  was  intended  to  address  the 
industrial  cost  problem  in  the  early  1980s. 
IMIP  had  a  two  pronged  approach: 

1)  to  improve  the  return  on  investment, 
bringing  it  up  to  market  levels,  so  that  a 
defense  firm  would  not  seek  alternative 
investment  opportunities  for  its  capital  (and 
also  be  willing  to  invest  in  more  efficient 
equipment,  increasing  profit  while  reducing 
cost  based  contract  pricing) 

2)  to  "guarantee"  amortization  of 
facilities  cost  over  a  negotiated,  program  based 
production  quantity.  If  less  than  the  number  of 
units  were  ordered,  the  Government  would 
recognize  a  pro  rata  share  of  the  shortfall 
between  the  net  asset  value  and  net  proceeds  of 
the  equipment's  sale  price. 
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EVUP's  concept  was  and  is  theoretically  sound; 
offering  high  potential  benefit  to  both 
contracting  parties.  However,  it  was  never  fully 
developed.  The  budgetary  increases  of  the 
Reagan  era  defense  buildup,  and  the  competing 
goals  of  CICA  (the  Competition  In  Contracting 
Act)  overcame  the  driving  force  for  detailed 
implementation.  As  a  result,  IMIP  has  not 
achieved  its  potential  and  the  DIB  issue  persists 
today.  Certain  initial  IMIP  policies  were 
considered  for  implementation,  but  not 
adopted,  that  would  have  mitigated  the  current 
scenario. 

Commercial/Defense  Industrial  Base: 
Distinction  or  Confusion? 

By  clearly  distinguishing  between  the  Defense 
and  the  Commercial  industrial  base,  DOD 
would  accept  responsibility  for  the  former,  and 
its  inability  to  control  or  to  interfere  with  the 
latter.  Appropriate  Government  management 
action  could  then  be  identified  (steps  to  be 
taken  or  precluded  for  each),  eliminating  the 
longstanding  ambivalence  that  results  when 
DOD  attempts  to  achieve  many  competing 
goals.  Examples  of  that  ambivalence  abound: 

a.  flowdown  of  Government  contract 
clauses,  cost  accounting  regulations  and 
oversight  to  a  commercial  base  that  offers 
some  products  that  are  better  and  cheaper,  but 
can  not  or  will  not  accept  Government 
controls. 

b.  obtaining  data  rights  so  that  the 
Government  can  compete  future  work,  and 
thereby  obtain  lower  prices,  while  Industry  will 
not  produce  intellectual  property  that  they  are 


not  properly  compensated  for,  from  their 
perspective.  The  firm  expending  the  effort  may 
not  only  be  denied  a  competitive  advantage, 
but  may  also  see  their  invention  transferred  to 
their  competitors. 

c.  CICA/competition  motivated  dual 
sourcing,  accommodated  during  the  Reagan 
defense  buildup,  produced  adverse  results  in 
today's  drawdown.  Today,  the  business  base 
for  many  systems  do  not  economically  support 
two  suppliers.  Defense  firms  or  business  units 
that  merge  have  been  allowed  to  retain  the  cost 
savings  resulting,  implying  a  cost  increase  due 
to  competition  when  instead,  sole  source 
would  have  been  economically,  but  not 
politically,  correct.  While  the  Federal  Trade 
Commission  has  objected  to  defense  contractor 
mergers  if  a  sole  source  situation  would  result, 
there  is  no  realistic  alternative.  If  dual  sources 
remain,  they  are  likely  to  be  weaker  from  both 
a  financial  and  technology/production 
capability  standpoint.  Accordingly,  over  the 
long  term,  the  intended  CICA  based  savings  to 
DOD  may  not  have  been  achieved. 

Defense  plants  will  not  be  available  if  they 
convert  to  non-military  markets,  or  worse,  if 
they  fail.  Nor  should  they  be  expected  to  be 
available  without  appropriate  financial  support. 
The  need  for  a  capable  defense  industrial  base 
is  only  tacitly  accepted,  but  is  not  actively 
supported.  The  essential  element  of  this 
problem,  on  the  macro  level,  is  that  DOD  does 
not  explicitly  recognize  its  financial 
responsibility  for  the  industrial  costs  that 
benefit  it.  On  the  micro  level,  there  is  no 
"contract"  for  a  contractor's  facility,  which 
would  specifically  recognize  these  industrial 
costs.  Instead,  it  is  a  contract  by-product; 
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without  responsibility  focus.  Seldom,  if  ever, 
are  industrial  costs  meaningfully  discussed  at 
contract  negotiations  with  acquisition  or 
contract  management  organizations;  even  in  the 
most  obvious  case,  when  a  single  program 
comprises  a  plant's  entire  business  base. 

Instead,  if  the  DOD/Commercial  industrial  base 
distinction  is  realistically  made,  DOD  can 
manage  the  former  more  effectively  and  utilize 
the  latter  in  a  way  that  it  has  not  before.  The 
defense  base  would  then  be  a  Government 
responsibility,  and  its  relationship  with 
industrial  firms  would  become  more  of  a 
partnership,  reducing  acrimony  and  optimizing 
shared  goal  attainment.  The  commercial  base, 
while  it  would  recognize  that  the  Government 
is  a  major  customer,  would  not  be  constrained 
in  delivering  products  that  will  compete  and 
win  in  what  is  becoming  more  of  a  world-wide 
marketplace  than  ever  before.  In  fact,  we 
should  recognize  that  not  only  would 
commercial  firms  generate  better  and  less 
costly  products,  but  also  the  income  which  can 
be  taxed  to  pay  for  the  defense  base. 

DOD's  DIB  implementation. 

Organizing  accordingly 

Once  DOD  accepts  that  responsibility,  it  should 
organize  accordingly,  by  industrial  sector  (e.g., 
satellites,  shipbuilding,  military  vehicles,  or 
munitions).  Service  acquisition  organizations 
are  already  organized  by  sector  (e.g.,  for 
aircraft,  the  USAFs  Aeronautical  Systems 
Division,  Naval  Air  Systems  Command  and 
Army  Aviation  Systems  Command);  but 
additional  mechanisms  may  be  required  for  a 


total  DOD  perspective  of  each  sector.  Once 
accomplished,  each  DOD  sector  would 
consolidate  existing  Service  time  phased 
requirements  for  systems:  for  their 
development,  production,  and  include 
surge/mobilization  considerations. 

The  sector  level  approach 

The  next  step  would  be  to  evaluate  the  existing 
dedicated  DIB  that  supports  each  sector,  to 
include  both  prime  and  subcontractors,  and 
which  may  overlap  with  other  sectors  to  some 
extent.  This  is  not  overly  burdensome,  as  only 
major  plants  would  be  involved,  most  which 
have  resident  contract  management 
organizations  already.  These  plants  have  a 
predominant  Government  business  base, 
comprised  of  a  single  or  of  many  programs. 
Once  those  plants  are  identified,  a  review  of 
each  plant  would  be  in  order  (for  example, 
"Should  Cost"  reviews  identify  those  plants 
that  are  not  operating  efficiently).  By  matching 
each  sector's  requirement  with  the  DIB 
supporting  it,  excess  capacity  would  be 
identified  as  well  as  the  basis  for  selecting  firms 
most  deserving  of  continued  support.  These 
decisions  would  not  be  entirely  subjective,  but 
would  include  quantitative  factors  summarized 
at  the  OSD  level.  Firms  not  selected  would 
then  be  forced  to  transition  to  the  commercial 
industrial  base,  during  which  they  would  be 
allowed  to  complete  their  current  contract 
work  or  to  sell  it  to  the  DIB  survivors  without 
FTC  challenge.  While  smaller,  the  resulting 
DIB  would  be  much  stronger,  and  the 
Commercial  Industrial  Base  would  similarly 
benefit. 
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DIB  contracting 

Many  other  improvements  would  result.  DIB 
prime  contractor  activities  would  more 
explicitly  recognize  their  systems  integrator 
role,  really  a  partnership,  enabling  program 
managers  to  perform  many  Government 
responsibilities  from  a  broader  and  deeper 
perspective.  Industrial  costs,  costs  for  such 
current  tasks  as  long  term  planning  or 
intellectual  property,  and  for  such  new  tasks  as 
identifying  and  evaluating  commercial  products 
which  could  be  utilized  in  weapons  systems 
(e.g.,  for  "dual  use"  initiatives)  are  expended 
for  the  mutual  benefit  of  both  partners.  Rather 
than  charge  "partnership"  costs  indirectly,  as 
currently,  they  would  be  charged  direct  to  a 
new,  separate  contracting  mechanism, 
collecting  costs  for  each  DIB  plant. 

For  example,  industrial  costs  and  many  of  the 
traditional  systems  integrator  activities  and 
responsibilities  would  be  reflected  in  a  separate 
contract  for  each  plant;  with  concomitant 
improvement  to  planning  and  programming 
activities.  The  costs  would  be  the  same  as 
before  (or  lower,  because  they  would  be 
directly  managed  by  accountable  individuals  in 
Government  and  Industry;  and  logically,  one 
would  assume  improved  information  and 
control  results  in  efficiency  and  effectiveness), 
but  charged  differently: 

An  appropriate  contract  type  could  be  cost  plus 
award  fee,  with  the  expectation  that  the 
Government  fee  determining  official  (EDO) 
would  probably  be  at  the  PEO  or  acquisition 
executive  level.  The  FDO  would  have  a  sector 
wide  and/or  DIB  wide  perspective,  and  would 


be  briefed  by  a  high  level  Industry  counterpart, 
so  the  expectation  would  be  that  plants  would 
be  appropriately  modernized  and  technology 
would  be  pursued  to  support  stated 
Government  objectives,  and  appropriately 
rewarded. 

Given  plant  costs  would  be  recognized 
separately,  weapons  system  unit  production 
costing  would  be  based  on  the  incremental 
quantity  desired,  removing  the  negative 
programmatic  aspects  cited  in  the  beginning  of 
this  article.  Because  of  the  partnership 
arrangement,  the  industry  partner  would  expect 
defined  future  production  work,  and  be 
allowed  access  to  now  denied  acquisition 
sensitive  information,  facilitating  joint  planning. 
Depending  on  the  situation,  costs  at  issue  could 
be  charged  to  the  plant  contract  or  to  the 
production  contract,  as  agreed  to  by  the 
parties. 

In  the  commercial  industrial  base,  Government 
controls  and  contractual  language  will  no 
longer  be  applicable.  Interdivisional  cost 
transfers  from  a  firm's  commercial  units  to  DIB 
units  would  be  accepted  by  the  Government  as 
certified  by  the  industrial  partner's  commercial 
accounting  firm,  and  cost  accounting  standards 
would  not  flow  the  other  way.  Currently, 
commercial  accounting  firms  certify  financial 
information,  subject  to  GAAP  standards  and 
Securities  and  Exchange  Commission 
regulation.  That  information  is  accepted  by  the 
investment  community,  and  could  be  accepted 
by  Government  acquisition  organizations  as 
well.  Given  some  extraordinary  issue,  DCAA 
could  be  allowed  entry  to  commercial  base  unit 
accounting  records  as  agreed  to  on  an 
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exception  basis  at  a  high  Govemment/Industry 
level. 

Summary 

It  is  clear  that  current  conditions  demand  this 
new  approach  be  pursued.  When  it  is,  the 
positive,  partnership  approach  espoused  by 
many  for  several  years  will  be  attained,  and  will 
persist.  Once  begun,  the  process  action  teams 
and  product  improvements  underway  will 


become  much  more  effective,  because  their 
scope  would  be  widened  within  targeted  DIB 
sectors.  This  initiative  should  be  combined 
with  a  similar  review  of  the  technology  base. 
Lastly,  with  relief  from  burdensome 
Government  regulations,  the  commercial 
industrial  base  could  deliver  lower  cost  and 
higher  quality  products  to  all  potential 
customers,  due  to  greater  economies  of  scale 
resulting  from  access  to  Govemment/Defense 
markets. 
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ABSTRACT 

Ordinal  scales  are  often  used  in  performing 
risk  and  System  Maturity  Matrix  (SMM) 
assessments  as  part  of  the  system 
engineering  process  in  defense  programs. 
However,  ordinal  scales  are  frequently  used 
incorrectly  in  these  assessments.  The  results 
obtained  can  be  meaningless,  thus 
undermining  the  effectiveness  of  the 
program's  system  engineering  process. 
Limitations  of  using  ordinal  scales  for  risk 
and  SMM  assessments  are  explored  in  this 
paper.  Examples  are  presented  from 
published  defense  program  risk  analysis  and 
SMM  methodologies  illustrating  common 
errors,  along  with  recommendations  to 
correct  these  errors. 

INTRODUCTION 

Aggressive  risk  management  by  both 
government  and  industry  is  generally 
essential  for  program  success.  Program 
decisions  and  resource  commitments  must  be 
based  on  plans  for,  and  progress  in, 
controlling  risk. 

Risk  analysis  (and  risk  assessment),  part  of 
risk  management,  and  System  Maturity 
Matrix  (SMM)  techniques  are  often  used  as 
part  of  the  overall  systems  engineering 
process.  Risk  areas  called  out  in  Department 


of  Defense  Directive  (DoDD)  5000.1  to  be 
assessed  at  milestone  decision  points  include: 
threat,  technology,  design  and  engineering, 
support,  manufacturing,  cost,  and  schedule 
[1].  The  SMM  is  a  tool  to  track  technical 
progress  and  risk.  It  links  user  specified 
requirements  to  specific  test  results  during 
system  development.  Actual  progress 
towards  requirement  verification  is  readily 
apparent  through  the  use  of  metrics  [2],  In 
risk  management  and  SMM,  the  program 
manager  is  given  broad  latitude  how  to 
implement  each  process.  (I  do  not  discuss 
the  implementation  processes  here  due  to 
space  limitations.) 

Ordinal  assessment  scales  are  often  used  to 
compute  the  level  of  risk  or  measure  the 
level  of  technical  progress  in  DoD  programs. 
Here,  the  resulting  numerical  value 
associated  with  risk  or  technical  progress  is 
dependent  upon  the  place  (e  g.,  first,  second, 
etc.)  it  occupies  in  the  ordered  sequence. 
The  numerical  value  of  ordinal  data  indicates 
its  relative  position  or  standing  [3], 

Although  ordinal  assessment  scales  can  be 
utilized  in  risk  analysis  and  SMM  techniques, 
the  scales  may  be  inherently  flawed,  and  the 
resulting  numerical  output  may  be 
meaningless.  For  example,  the  program 
manager  may  wish  to  fund  one  or  more 
potential  risk  reduction  options,  but  the 
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prioritized  list  he  evaluates  may  have  very 
little  value  if  improperly  derived. 

I  examine  in  this  paper  some  common  errors 
associated  with  using  ordinal  scales  in  risk 
analysis  and  SMM,  and  offer  possible 
solutions  to  these  problems.  These  errors 
are  not  simply  drawn  from  theoretical 
considerations,  but  from  actual  experience 
obtained  on  several  large-scale  Department 
of  Defense  programs  over  the  past  decade. 
Moreover,  many  of  the  ordinal  scale 
problems  I  will  discuss  are  applicable  to  a 
broad  range  of  systems  engineering 
applications  beyond  risk  analysis  and  SMM. 

I  provide  in  the  remainder  of  this  paper  a 
definition  of  risk,  then  present  and  discuss 
limitations  associated  with  using  ordinal 
scales  in  risk  analysis  and  SMM,  some 
general  issues  that  affect  the  ability  to 
perform  these  assessments,  and  briefly 
discuss  two  published  risk  management 
examples  that  incorrectly  use  ordinal  scales. 

THE  MATHEMATICAL 

REPRESENTATION  OF  RISK 

Risk  is  defined  in  the  Defense  Systems 
Management  College  (DSMC)  Systems 
Engineering  Management  Guide  as  the: 
"combined  effect  (product)  of  the  likelihood 
of  the  occurrence  and  a  measured  or 
assessed  consequence  given  that 
occurrence."  [4]  The  likelihood  of 
occurrence  refers  to  the  probability  of  the 
event  taking  place.  Similarly,  the  DSMC 
Risk  Management  Concepts  and  Guidance 
document  defines  risk  as  the  product  of  "the 
probability  of  an  undesirable  event  occurring 
and  the  significance  of  the  consequence  of 
the  occurrence."  [5] 

The  word  "risk"  is  often  used  incorrectly  to 
represent  only  the  probability  term.  The 


correct  usage  is  risk  as  the  product  of  the 
probability  and  consequence  terms.  The 
word  uncertainty  is  appropriate  for  ordinal 
scales,  which  clearly  do  not  represent 
probabilities.  (I  use  the  term  "risk 
(uncertainty)"  in  the  rest  of  this  paper  to 
indicate  instances  of  uncertainty  where  the 
term  "risk"  is  generally  incorrectly  used.) 

Ordinal  assessment  scales  (discussed  in  the 
next  section)  can  be  used,  so  long  as  it  is 
recognized  their  values  are  not  probabilities 
and  they  cannot  be  used  to  calculate  risk  (as 
defined  above). 

Ordinal  assessment  scales  may  express  the 
level  of  uncertainty  for  the  item  being 
evaluated.  They  are  potentially  well  suited 
for  the  DoDD  5000.1  threat,  technology, 
design  and  engineering,  manufacturing,  and 
support  risk  assessment  categories,  plus 
other  potential  categories  deemed  necessary 
by  the  program  manager  (e.g., 
hardware/software  integration). 

Quantitative  cost  and  schedule  risk 
assessment  methodologies  can  ideally 
estimate  risk  since  probabilities  of 
occurrence  result  from  the  simulation 
process  and  the  consequence  term  directly 
represents  either  cost  (dollars)  or  schedule 
(time).  (Ordinal  scales  are  sometimes 
misused  in  establishing  characteristics  of  cost 
or  schedule  probability  distributions  at  the 
Work  Breakdown  Structure  level.  However, 
I  do  not  discuss  this  issue  here  due  to  space 
limitations.) 

Given  this  introduction,  it  is  beneficial  to 
examine  appropriate  definitions  of 
uncertainty  applicable  to  program  risk 
categories. 

There  are  typically  five  classes  of 
uncertainty:  (1)  certainty;  (2)  probability 
distributions  of  known  form  embedded  in 


456 


known  models,  covering  known  possible 
states;  (3)  probability  distributions  of 
unknown  form  embedded  in  known  models; 
(4)  uncertain  models  (strong  uncertainties); 
and  (5)  chaos  [6],  The  military  system 
development  process  is  typically 
representative  of  states  (3)  and  (4). 
Imperfect  information  typically  exists  for  a 
given  set  of  program  risk  category  (e.g., 
threat)  attributes;  hence  states  (1)  and  (2)  are 
unlikely.  Conversely,  some  knowledge  of 
possible  risk  category  trends  typically  exists; 
hence  state  (5)  is  unlikely. 

Ordinal  assessment  scales,  which  do  not 
require  assumptions  about  probability 
distributions,  are  compatible  with  states  (4) 
and  possibly  (3)  are  potentially  well  suited 
for  risk  categories  such  as  threat, 
technology,  design  and  engineering, 
manufacturing,  and  support.  This  is  not  to 
say  that  ordinal  scales  are  without  defect  or 
that  mathematical  operations  can  be 
performed  on  the  results  (as  discussed  in  the 
next  section). 

However,  the  basis  for  quantitative  analytic 
or  stochastic  risk  assessments,  including  cost 
and  schedule,  is  state  (2),  where  probability 
distributions  of  known  form  are  embedded  in 
known  models,  covering  known  possible 
states.  A  match  is  unlikely  to  occur  between 
the  true  program  probability  distributions 
and  those  used  in  the  risk  analysis  procedure. 
Thus,  even  the  results  of  quantitative  risk 
assessments  are  likely  to  be  somewhat 
unreliable. 

SOME  LIMITATIONS  OF  USING 
ORDINAL  SCALES 

Some  Inherent  Limitations  of  Ordinal 
Scales 

A  number  of  caveats  exist  for  using  ordinal 
scales  in  the  risk  analysis  process.  First, 


ordinal  scales  values  represent  rank 
orderings.  For  example,  the  fifth  scale  level 
(value  of  5)  is  greater  than  the  fourth  scale 
level  (value  of  4).  Second,  ordinal  scale 
value  are  generally  not  absolute.  For 
example,  the  lowest  values  does  not 
necessarily  represent  a  cardinal  "0"  value  or 
any  other  value.  There  is  no  valid  rationale 
to  assign  decimal  values  to  uncalibrated 
ordinal  scale  levels,  since  the  scale  levels  are 
ordinal,  not  cardinal,  in  nature.  In  addition, 
providing  decimal  scale  level  scores  (e.g., 
0.2)  suggests  a  degree  of  accuracy  that  is 
unwarranted  and  invites  the  user  to  perform 
mathematical  operations  which  will  yield 
erroneous  results  (as  discussed  below). 
Third,  ordinal  scales  generally  have  arbitrary 
intervals  between  scale  levels.  For  example, 
the  difference  between  levels  5  and  4  is  likely 
not  the  same  as  between  levels  4  and  3. 
Providing  linear,  logarithmic,  exponential,  or 
other  transformations  for  uncalibrated 
ordinal  scale  levels  will  yield  meaningless 
results.  Fourth,  ordinal  scales  are  monotonic 
and  positive  in  nature— meaning  that  the 
uncertainty  associated  with  level  5  is  greater 
than  level  4,  which  is  greater  than  level  3, 
etc.  However,  as  mentioned  in  the  third  item 
above,  the  degree  to  which  level  5  is  greater 
than  level  4  is  generally  unknown. 

Hence,  while  the  scale  level  assigned  to  a 
given  item  (e.g.,  subsystem)  may  be  correct 
when  evaluated  for  a  given  risk  or  SMM 
category,  the  associated  numerical  value  is 
generally  not  meaningful  beyond  an  ordering 
sense  unless  the  scales  are  calibrated.  For 
example,  the  best  that  can  be  said  of  an  score 
of  5  is  that  it  is  greater  than  a  score  of  2.  It 
is  not,  however,  2.5  times  greater  than  a 
score  of  2,  or  generally  any  other  known 
factor,  but  simply  greater  than  2. 

With  uncalibrated  scales,  it  is  not  possible  to 
perform  any  mathematical  operations  with 
any  degree  of  confidence  unless  the  ordering 
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of  the  sums  or  products  is  maintained  at  all 
levels.  However,  even  then  the  resulting 
quantitative  values  will  only  represent  rank 
ordering  between  items.  For  example,  if  two 
ordinal  scales  are  used  and  item  1  receives 
scores  of  5  and  4  for  the  two  scales  and  item 
2  receives  scores  of  4  and  3  for  the  same  two 
scales,  the  best  that  can  generally  be  said  is 
that  item  1  has  a  higher  level  of  uncertainty 
than  item  2. 

Values  Are  Not  Probabilities 

Ordinal  scale  values  are  not  probabilities, 
which  are  also  cardinal  values.  Hence,  any 
mathematical  manipulation  between 
categories  (e.g.,  technology  and 
manufacturing)  or  subcategories  (e.g., 
manufacturing  process  difficulty  and 
production  equipment  status  within 
manufacturing)  is  not  statistically  rigorous 
since  the  original  values  are  not  probabilities. 

Ordinal  scale  values  are  not  probabilities 
simply  because  the  developer  of  the  scale 
uses  decimal  scale  value  scores  and/or 
declares  the  values  to  be  probabilities! 

Given  multiple  ordinal  scales,  the  analyst  or 
program  manager  is  often  tempted  to  roll-up 
the  scores  mathematically  into  a  single  value. 
I  have  examined  a  variety  of  normalization 
schemes  and  averaging  techniques— none  of 
them  yield  meaningful  results.  I  have  also 
observed  a  variety  of  other  mathematical 
operations  performed  on  uncalibrated  ordinal 
scores,  ranging  from  simple  algebraic 
manipulation  to  representing  coefficients  of 
partial  derivatives.  Again,  the  results  are 
meaningless. 

Techniques  exist  that  may  be  used  to 
calibrate  ordinal  scales,  such  as  the 
Analytical  Hierarchy  Process  (AHP)  [7], 
(The  calibration  of  scale  levels  within  a 


single  ordinal  scale  can  be  viewed  as  vertical 
calibration.  Calibration  (weighting)  between 
scales  can  be  viewed  as  horizontal 
calibration.)  While  this  may  provide  a 
relative  calibration  within  a  scale  for  different 
levels,  or  between  scales,  it  will  not 
absolutely  calibrate  any  given  scale  level  or 
weighting  between  scales.  For  example,  if 
the  fifth  technology  scale  level  has  a 
calibrated  score  of  4.6  and  the  fourth  scale 
level  has  a  calibrated  score  of  2.3,  then  the 
fifth  scale  level  is  2.0  times  greater  than  the 
third  scale  level.  However,  the  calibrated 
scores  are  not  probabilities,  and  the  absolute 
magnitude  of  any  scale  level  generally 
remains  unknown.  For  example,  the  lowest 
scale  level  does  not  represent  zero 
probability  or  uncertainty  in  almost  any 
instance. 

In  addition,  it  is  not  advisable  to  perform 
mathematical  operations  on  scales  with 
relative  calibration  because  the  meaning  of 
the  results  may  be  difficult  to  interpret.  For 
example,  if  three  categories  are  used;  such  as 
technology,  design  and  engineering,  and 
manufacturing;  and  their  respective 
calibrated  scale  scores  for  a  particular  item 
are  6.0,  1.5,  and  0.5,  it  is  generally  of  greater 
interest  to  note  each  score,  than  the  sum, 
average,  or  other  combination  of  the  three 
scores  which  tends  to  mask  the  relatively 
high  technology  score. 

In  general,  it  is  recommended  that  raw 
ordinal  scores  be  used  in  the  program's  risk 
management  and  SMM  tracking  process. 
When  a  single  score  is  needed  per  category, 
then  the  highest  of  the  subcategory  scores 
should  be  used.  Although  this  represents  a 
conservative  approach,  it  prevents  potentially 
noteworthy  issues  from  being  masked  and 
overlooked. 

Inconsistent  Ordinal  Scale  Definitions 
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Lengthy  ordinal  scale  level  definitions  should 
be  avoided  since  they  provide  the  possibility 
of  multiple  attributes  per  level  or  ambiguities 
for  a  given  level  or  between  levels.  For 
example,  if  three  separate  attributes  must  be 
met  to  move  from  a  score  of  3  to  2,  how 
should  the  analyst  interpret  meeting  one  of 
the  three  criteria  or  two  of  the  three  criteria, 
but  not  all  three?  While  the  conservative 
approach  is  to  require  all  criteria  be  met  to 
achieve  the  next  lowest  score  (in  this  case 
going  from  a  3  to  2),  it  may  well  be  better  to 
focus  the  ordinal  scale  definition  to  a  single 
attribute,  and  provide  clarifying  information 
in  a  supplementary  write-up. 

Similarly,  as  the  number  of  scale  definition 
attributes  increases,  so  does  the  possibility  of 
having  ambiguities  for  a  given  scale  level  or 
between  two  or  three  adjacent  scale  levels. 
For  example,  if  three  attributes  exist  for  scale 
levels  4  and  5  it  may  be  possible  that  one  or 
more  of  the  attributes  for  scale  level  4  may 
actually  imply  a  higher  level  of  uncertainty 
than  one  or  more  attributes  for  scale  level  5, 
thus  violating  the  underlying  assumption  that 
the  scale  is  monotonic  and  positive  in  nature. 

A  definition  with  generally  less  than  10 
words,  or  certainly  less  than  15  words,  is 
likely  sufficient  to  convey  the  intent  of  the 
scale  level  without  creating  a  multi-attribute 
or  ambiguous  definition.  Additional 
information  to  convey  nuances  and  clarify 
the  scale  level  definitions  can  be  provided 
separately.  One  or  two  word  scale 
definitions  should  also  be  avoided  since  they 
are  too  subjective  even  if  supplementary 
material  is  available. 

How  Many  Ordinal  Scale  Levels  Are 
Appropriate? 

Three  scale  levels  are  generally  too  few, 
while  10  or  more  levels  are  generally  too 


many.  In  the  former  case  there  is  generally 
too  little  granularity,  while  in  the  latter  case 
too  much.  The  tendency  when  only  three 
levels  are  used  is  to  automatically  assume 
they  represent  high,  medium,  and  low 
uncertainty,  which  may  not  necessarily  be  the 
case.  When  a  large  number  of  scale  intervals 
are  used  the  possibility  exists  that 
ambiguities  between  two  or  three  adjacent 
scale  level  definitions  may  occur.  In 
addition,  a  large  number  of  scale  levels  are 
generally  inappropriate  unless  a  well  defined, 
repeatable  process  exists  that  is  being 
evaluated. 

Some  Ordinal  Scale  Characteristics 

Many  ordinal  risk  (uncertainty)  and  SMM 
scales  are  time-event  based,  and  their  values 
represent  a  state  of  maturity.  That  is,  higher 
scores  generally  represent  items  or  programs 
in  early  stages  of  development,  while  lower 
scores  generally  represent  more  mature 
events,  including  production  programs  or 
deployed  items.  (Of  course,  the  scale 
ordering  or  associated  numbering  can  be 
reversed  if  desired.)  This  is  generally  the 
case  for  ordinal  technology,  design  and 
engineering,  and  manufacturing  scales.  It 
may  be  the  case  for  ordinal  support  scales.  It 
may  also  be  implied  in  ordinal  threat,  cost, 
and  schedule  scales.  An  inherent,  and 
generally  valid,  assumption  in  using  such 
ordinal  scales  is  that  the  item  being  evaluated 
generally  follows  the  requisite  time-varying 
maturity,  and  not  an  arbitrary  order  (even 
though  some  fluctuations  may  occur). 

Possible  Correlation  Between  Scale  Sub- 
Categories  and  Categories 

The  relationship  between  ordinal 
subcategory  scales  for  a  given  category  (e.g., 
manufacturing  process  difficulty  and 
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production  equipment  status)  or  between 
categories  (e.g.,  technology  and  design  and 
engineering)  may  not  be  uncorrelated.  This 
is  often  true  at  the  highest  and  lowest  scale 
levels,  which  imply  for  example,  scientific 
research  and  operational  systems, 
respectively.  If  the  subcategories  or 
categories  are  correlated  with  each  other, 
then  they  are  not  statistically  independent. 
Statistical  independence  may  be  a  necessary 
assumption  for  performing  some 
mathematical  operations  on  ordinal  scale 
values— particularly  if  the  erroneous 
assumption  is  made  that  scale  values  are 
probabilities. 

However,  even  if  correlation  coefficient(s) 
between  scales  can  be  identified,  they  cannot 
be  used  in  any  meaningful  way  to  adjust  the 
numerical  results  because  of  the  ordinal,  non¬ 
probability  limitations  present. 

Ordinal  Scale  Titles 

Calling  ordinal  scales  "confidence"  scales  as 
done  in  some  instances  is  misleading— the 
same  limitations  apply  to  the  scales 
regardless  of  their  title.  In  addition,  there  is 
no  relationship  between  these  scales  and  the 
statistical  confidence  interval,  as  discussed 
next. 

Absence  of  Confidence  Intervals 

No  confidence  intervals  around  a  given  risk 
(uncertainty)  or  SMM  score  can  be 
generated  because  of  the  ordinal  and  non¬ 
probability  nature  of  the  scales. 

Incompleteness  of  Scale  Categories 

The  ordinal  scale  sub-categories  may  not 
provide  an  extensive  representation  of  the 
uncertainty  contributors  for  a  given  risk  or 
SMM  category.  For  example,  a  single 


technology  scale  may  represent  a  component 
of  technology  risk  (uncertainty),  but  it  is  not 
likely  an  all-inclusive  descriptor  of  this  risk 
(uncertainty).  Implementing  additional  sub¬ 
categories  will  may  increase  the  difficulty  of 
interpreting  and  rolling-up  the  resulting 
scores  because  of  the  ordinal  nature  of  this 
methodology  than  if  a  cardinal  methodology 
(e.g.,  probability-based)  is  used. 

Risk  Categories 

The  DoDD  5000.1  risk  categories  should  be 
viewed  as  the  minimum  number  of  risk 
categories  to  be  evaluated,  although  the 
threat  category  may  diminish  in  importance 
in  the  1990s  for  some  defense  systems  (e.g., 
strategic).  (All  or  a  subset  of  these 
categories  should  be  used  in  the  SMM  for 
tracking  maturity.)  The  technology  and 
design  and  engineering  risk  categories  may 
include  both  hardware  and  software 
subcategories.  Other  risk  categories  can  be 
developed  and  used  at  the  discretion  of  the 
program  manager.  For  example,  risk 

assessments  for  hardware/software 
integration  may  be  warranted  in  some  cases. 

Some  risk  categories  may  be  misnamed,  and 
actually  represent  another  risk  category.  For 
example,  software  manufacturing  should  be 
designated  as  software  design  and 

engineering.  Other  risk  categories  may  be 
inappropriate  or  inherently  ambiguous  and 
should  be  eliminated.  This  includes 
categories  such  as  dependence/correlation 
(which  affects  the  relationship  between 
categories),  requirements  (perhaps  too 

broad),  complexity  (too  vague),  reliability 
(better  treated  as  a  technology  or  design  and 
engineering  consideration),  alternate  item 

availability  (better  evaluated  by  rating  the 
alternate  item(s)),  and  criticality  to  mission 
(actually  related  to  the  consequence  term). 
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The  number  of  subcategories  per  category 
can  have  the  undesirable  impact  of  weighting 
the  importance  of  a  given  category.  For 
example,  if  threat  risk  is  represented  by  7 
subcategories  (with  one  scale  per 
subcategory)  while  technology  risk  is 
represented  by  only  a  single  scale,  no  greater 
level  of  importance  should  be  placed  on 
threat  risk  than  technology  risk  (all  else  held 
constant).  The  true  number  of  subcategories 
per  category  may  be  masked  in  cases  where 
the  DoDD  5000.1  risk  categories  are  not 
clearly  delineated.  For  example,  in  one  set  of 
published  ordinal  risk  (uncertainty)  scales, 
statistical  process  control,  producibility,  and 
manufacturing  precision  are  listed  as  risk 
categories,  yet  all  are  related  to  the 
manufacturing  risk  category.  In  this  set  of 
scales,  technology  (Technology  Status), 
design  and  engineering  (Required  Technical 
Advancement),  cost  (Cost),  and  schedule 
(Schedule)  were  represented  by  a  single 
scale.  Thus,  in  this  case  the  analyst  may 
erroneously  assume  that  manufacturing  is  the 
dominant  risk  category,  when  this  is  not 
necessarily  true. 

If  multiple  subcategories  exist,  particularly  if 
derived  from  uncalibrated  ordinal  scales,  a 
single  score,  generally  the  highest  of  the 
subcategory  scores,  should  be  reported  at  the 
summary  level  to  avoid  the  appearance  of 
importance  being  related  to  the  number  of 
subcategories  per  risk  category. 

Biased  results  will  still  likely  exist  when 
multiple  subcategories  are  present  even  if 
vertically  calibrated  ordinal  scales  are  used. 
This  will  be  the  case  unless  a  horizontal 
calibration  is  performed  between 
subcategories  per  risk  category  and  between 
risk  categories  by  a  suitable  technique  (e.g., 
AHP,  but  note  the  potential  limitations 
discussed  above). 


Consequence  of  Occurrence  Term 

While  the  probability  risk  term  is  bounded  by 
0  to  1,  the  consequence  risk  term  can  range 
from  zero  to  infinity— a  much  greater  range. 
When  ordinal  consequence  scales  are  used 
the  resulting  compression  versus  cardinal 
consequence  may  easily  reach  several  orders 
of  magnitude  (e.g.,  a  cardinal  factor  of  500 
versus  5  ordinal  levels)  making  interpretation 
by  decision  makers  difficult. 

Given  an  ordinal  or  cardinal  consequence 
score,  there  is  an  inherent  danger  of  the 
analyst  or  program  manager  multiplying  it  by 
an  ordinal  uncertainty  value  in  an  attempt  to 
obtain  risk.  An  ordinal  or  cardinal 
consequence  term  cannot  be  multiplied  with 
a  calibrated  or  uncalibrated  ordinal 
uncertainty  scale  score  to  obtain  risk.  This  is 
because  risk  requires  the  uncertainty  term  to 
be  a  probability,  and,  strictly  speaking,  the 
consequence  term  to  be  a  cardinal  number. 
Consequently,  even  if  a  calibrated  ordinal 
uncertainty  and  consequence  scales  are  used, 
the  resulting  product  value  is  not  risk. 

As  in  the  uncertainty  dimension,  a  single 
ordinal  scale  or  descriptor  may  not  be 
sufficient  in  most  cases  to  capture  the  impact 
of  consequence  of  occurrence.  For  example, 
while  dollars  and  time  are  often  suitable 
consequence  descriptors  for  cost  and 
schedule  risk,  no  single  consequence  scale  or 
simple  descriptor  is  likely  possible  for  other 
risk  categories,  including,  threat,  technology, 
design  and  engineering,  manufacturing,  and 
support.  Multiple  descriptors  or 

consequence  scales  may  be  needed  to 
properly  assess  the  impact  on  the  overall 
vehicle  mission. 

Some  candidate  consequence  scales  for 
aircraft,  launch  vehicles,  missiles,  and 
spacecraft,  might  include:  anomalous  item 
severity  (the  impact  the  anomaly  has  on  the 
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specific  hardware  item),  functional  effectivity 
(the  impact  the  anomaly  has  on  the 
performance  capability  of  various  levels  of 
vehicle  assembly),  mission  effectivity  (the 
effect  the  anomaly  has  on  the  overall 
system),  and  mission  impact  severity  (the 
impact  the  anomaly  has  on  the  vehicle's 
ability  to  fulfill  its  mission  objectives). 

TWO  EXAMPLES  OF 

INAPPROPRIATE  USE  OF  ORDINAL 
SCALES 

I  will  briefly  summarize  some  errors 
pertaining  to  the  use  of  ordinal  scales  for  risk 
management  found  in  two  DSMC 
publications. 

The  first  case  involves  software  risk  analysis 
scales  developed  by  Air  Force  Systems 
Command  [8],  which  appeared  in  the  DSMC 
Risk  Management  Concepts  and  Guidance 
[9],  The  ordinal  scales  had  three  levels  (too 
few),  the  levels  were  inappropriately  given 
decimal  values  (0.0-0. 3,  0.4-0. 5,  0.6-1. 0), 
and  the  levels  were  incorrectly  termed 
probabilities.  Many  of  the  scale  definitions 
were  too  short  (e.g.,  verified  projections), 
subjective  (e.g.,  little  or  none),  or  contained 
multiple  attributes  (e.g.,  documented, 
validated,  in  place). 

The  second  case  involves  risk  analysis  scales 
which  appeared  in  the  second  edition  of  the 
DSMC  Systems  Engineering  Management 
Guide  [10],  (This  material  was  deleted  in  the 
subsequent  edition.)  The  ordinal  scale  levels 
were  inappropriately  given  decimal  values 
(0.1,  0.3,  0.5,  0.7,  0.9)  and  the  levels  were 
incorrectly  treated  as  probabilities  in  some 
cases.  Many  of  the  scale  definitions  were 
too  short  (e.g.,  existing),  subjective  (e.g., 
significant  increase),  or  contained  multiple 
attributes  (e.g.,  technology  available, 
complex  design).  Invalid  mathematical 


equations  were  given  to  derive  an  overall 
probability  of  failure,  consequence  of  failure, 
and  risk  factor.  (The  underlying  ordinal 
numbers  cannot  be  mathematically 
manipulated.)  The  complexity  factor 
"probability"  category  may  actually  represent 
design  and  engineering.  The  dependency 
factor  "probability"  category  is  inappropriate 
(affects  the  relationship  between  categories). 
Finally,  portions  of  the  cost  factor  scale 
represent  risk  rather  than  consequence  (e.g., 
cost  estimates  increased  by  5  to  20  percent). 

CONCLUSIONS 

Risk  analysis  and  SMM  are  potentially 
important  defense  program  system 
engineering  tools.  Ordinal  scales  are  often 
used  to  quantify  the  level  of  risk  or  maturity 
present  in  these  assessments.  However,  a 
broad  range  of  errors  may  result  due  to 
assumptions  associated  with  the  nature  of  the 
ordinal  scales.  These  errors  can  lead  to 
meaningless  results  which  may  adversely 
affect  the  integrity  of  the  program's  systems 
engineering  process. 

Common  errors  associated  with  the  use  of 
ordinal  scales  include  assumptions  that  the 
resulting  numbers:  are  cardinal,  represent 
probabilities  (for  the  non-consequence  term), 
and  can  be  mathematically  manipulated. 

Values  obtained  from  ordinal  scales  are  not 
cardinal,  and  the  absolute  magnitude  of  the 
scale  levels  may  remain  unknown  even  after 
calibration.  In  addition,  true  risk  values 
cannot  be  generated  from  ordinal  scales, 
since  risk  is  the  product  of  likelihood  of 
occurrence  (probability)  and  consequence  of 
occurrence  terms.  Finally,  results  obtained 
from  mathematical  operations  are  generally 
inaccurate  for  uncalibrated  ordinal  scales, 
and  may  not  be  meaningful  even  when  the 
scales  are  calibrated. 
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Ordinal  scales  may  also  contain  substantial 
flaws,  including:  inconsistent  scale  level 
definitions,  an  inappropriate  number  of 
levels,  correlation  between  different 
categories,  and  the  incomplete  nature  of 
scales  representing  a  given  risk  category. 

Finally,  some  potential  problems  associated 
with  risk  categories  and  the  consequence  of 
occurrence  term  often  exist.  In  the  risk  case 
the  tendency  is  to  include  inappropriate 
categories,  or  too  many  subcategories  per 
risk  category.  In  the  consequence  case  the 
inclination  is  to  use  only  a  single 
consequence  of  occurrence  category  when 
more  than  one  may  be  desirable,  if  not 
necessary. 
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ABSTRACT 

Recent  environmental  legislation  passed  by 
the  U.S.  Congress  is  forcing  industry  to 
address  pollution  prevention  during  the 
production  process.  Companies  selling  to  the 
Department  of  Defense  are  not  exempt  from 
these  requirements.  This  paper  examines  the 
acquisition  process  from  Phase  0  (Concept 
Exploration  &  Definition)  to  Phase  IV 
(Operations  &  Support)  in  an  effort  to 
determine  where  in  the  acquisition  process 
pollution  prevention  activities  can  be  most 
cost-effective.  A  methodology  is  then 
presented  which  allows  for  the  evaluation  of 
products  from  an  environmental  approach. 
This  approach  is  known  as  Life  Cycle 
Assessment  (LCA)  and  provides  an  estimate 
of  the  total  environmental  burden  of  a 
weapon  system.  An  LCA  evaluates  the 
environmental  releases  and  impacts  of  a 
product/process/activity  by  tracking  its 
development  from  raw  materials  to  through 
production  and  ultimately  to  demilitarization 
or  disposal. 

INTRODUCTION 

The  Congress  of  the  United  States  enacted 
Public  Law  101-508  on  November  5,  1990. 
This  law  is  also  known  as  the  Pollution 
Prevention  Act  of  1990.  The  intent  of  this 
law  is  to  force  all  industry  to  reduce  or 
prevent  pollution  at  the  source,  i.e.,  during 
the  production  process.  For  those  companies 
involved  in  Department  of  Defense  activities, 
this  is  during  the  weapon  system  acquisition 
process.  The  acquisition  process  includes  the 
development,  production,  operation,  and 
final  disposition  of  all  weapon  systems 
procured  by  the  Department  of  Defense 
(DoD).  Also  included  in  this  concept  are  the 
raw  materials  used  in  the  production  and 
maintenance  of  weapon  systems.  By 
addressing  the  issue  of  pollution  prevention 
early  in  the  acquisition  cycle,  cost-effective 
changes  can  be  made  to  a  system  to  allow  it 
to  comply  with  all  environmental  laws.  This 


will  have  the  effect  of  reducing  the  potential 
for  costly  liability  settlements  by  the  federal 
government  as  well  as  protecting  the 
environment. 

In  order  to  be  most  effective,  an 
environmental  philosophy  must  be  implanted 
in  the  acquisition  process  beginning  with 
Phase  0,  Concept  Exploration.  During  this 
phase,  the  types  of  considerations  should 
include  the  potential  environmental  impacts  of 
the  various  options  being  considered.  It  is 
during  this  phase  that  the  most  cost-effective 
pollution  prevention  design  decisions  can  be 
made.  In  Phase  I,  Demonstration  & 
Validation,  system  performance  and 
environmental  impact  trade-offs  must  be 
made.  This  is  the  final  phase  in  which  the 
window  of  opportunity  for  cost-effective 
pollution  prevention  solutions  remains 
completely  open.  Beginning  with  Phase  II, 
Engineering  &  Manufacturing  Development, 
opportunities  for  pollution  prevention  begin 
decreasing  because  the  weapon  system 
design  is  frozen  and  the  issues  shift  to  those 
of  control  of  harmful  substances  and 
compliance  with  applicable  laws. 

DISCUSSION 

Pollution  prevention  is  quickly  becoming 
more  than  a  desirable  goal.  It  is  approaching 
the  status  of  a  necessity.  Clean-up  costs  are 
skyrocketing.  Hazardous  waste  disposal  and 
liability  costs  have  increased  tenfold  over  the 
past  five  years.  Landfill  space  has  dwindled 
80%  over  the  past  decade  (1).  Even  with 
total  compliance  with  all  regulatory  laws  and 
policies,  the  lack  of  physical  space  for 
disposition  of  wastes  cannot  be  ignored. 
Siting  for  new  landfills  or  hazardous  waste 
facilities  has  become  extremely  contentious. 
Not-in-my-backyard  (NIMBY)  has  become 
the  rallying  cry  of  those  individuals  and 
groups  opposing  new  landfills  or  other  types 
of  waste  facility.  Examples  of  this  situation 
abound.  A  recent  article  in  The  Wall  Street 
Journal  describes  the  situation  in  San  Diego, 
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California,  where  a  group  of  homeowners  is 
suing  San  Diego  County  for  $20  million 
because  the  county  plans  to  expand  an 
existing  landfill,  a  landfill  that  existed  before 
most  of  the  homeowners  moved  into  the  area. 
What  makes  this  case  somewhat  unique  is 
that  the  landfill  is  not  in  a  poor  or  minority 
neighborhood.  It  is  located  in  one  of  the 
county’s  most  affluent  areas  (2).  Similar 
situations  will  almost  certainly  occur  in  many 
other  parts  of  the  country.  To  minimize 
these,  systems  must  be  designed  that  avoid  or 
reduce  disposal  requirements. 

Most  current  environmental  programs  deal 
with  the  end  of  the  acquisition  pipe.  Clean¬ 
up  and  compliance  policies  and 


programs  fall  into  this  situation.  To  a  certain 
extent,  conservation  programs  do  also.  This 
is  the  least  desirable  and  most  costly  option  to 
addressing  environmental  pollution. 
Pollution  prevention,  on  the  other  hand,  deals 
with  environmental  protection  at  the  other  end 
of  the  acquisition  pipeline  -  during  Phase  0 
(Concept  Exploration)  and  Phase  I 
(Demonstration/  Validation).  Once  a  program 
has  progressed  into  Phase  II  (Engineering 
and  Manufacturing  Development),  emphasis 
necessarily  shifts  to  pollution  control, 
compliance  with  appropriate  laws,  and 
ultimately,  clean-up  and  disposal. 
Environmental  considerations  during  each 
stage  of  the  acquisition  cycle  are  shown  in 
Figure  1. 
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An  ounce  of  prevention  is  worth  a  pound  of 
cure.  Everybody  has  heard  this  old  Ben 
Franklin  adage  many  times.  If  an  ounce  of 
prevention  is  in  fact  worth  a  pound  of  cure, 
that's  a  16  to  1  benefit/cost  ratio.  Is  this 
platitude  applicable  in  today's  world  when 
addressing  problems  such  as  pollution 
prevention?  Examples  from  a  number  of 
companies  have  shown  that  this  is  possible. 
Forty  documented  case  studies  have  shown 
that  pollution  prevention  is  not  only  good  in 
terms  of  protecting  the  environment,  but  also 
in  terms  of  dollars  and  cents  (3).  The  firms 
involved  in  the  case  studies  were  grouped  by 
Standard  Industrial  Classification  (SIC) 
codes  and  encompassed  agriculture,  mining, 
food  products,  textiles,  furniture,  paper  and 
printing,  chemicals,  electronic  and  electrical 
equipment,  and  service  industries.  The 
results  are  impressive.  By  adopting  pollution 
prevention  strategies,  the  firms  achieved  the 
following: 

•  13  firms  improved  productivity, 

•  26  firms  reduced  raw  material  usage, 

•  13  firms  improved  energy  costs, 

•  17  firms  reduced  pollution  control  costs, 

•  7  firms  improved  product  quality. 

While  not  all  of  the  SIC  codes  covered  in  the 
case  studies  are  applicable  to  the  defense 
industry,  certainly  those  in  the  SICs  for 
chemicals,  electronic  and  electrical 
equipment,  and  service  industries  are 
appropriate. 

Can  these  cost  savings,  already  recognized 
by  private  sector  firms,  be  realized  in  the 
systems  acquired  by  DoD?  The  federal 
government  and  the  Department  of  Defense 
apparently  feel  that  the  answer  to  this 
question  is  "Yes".  DoD  is  getting  very 
serious  about  pollution  prevention.  How 
serious  are  they?  Serious  enough  to  create  a 
program  and  position  at  the  deputy  under 
secretary  of  defense  level.  The  secretary  of 
defense  has  created  the  DoD  Environmental 
Security  Program  and  placed  the 
responsibility  for  this  program  under  the 
Office  of  the  Deputy  Under  Secretary  of 
Defense  (Environmental  Security) 
(ODUSD(ES)).  The  mission  of  this  office  is 
to  integrate  DoD  environmental  concerns  and 
policies.  To  this  end,  DoD  has  developed  a 


number  of  directives  and  instructions  dealing 
with  pollution  prevention,  including  DODD 
4210.15,  Hazardous  Material  Pollution 
Prevention,  and  a  revised  DoD  5000  series 
dealing  with  systems  acquisition.  DoD 
Instruction  5000.2,  Defense  Acquisition 
Management  Policies  and  Procedures,  dated 
February  23,  1991  typifies  this  increased 
environmental  emphasis.  In  this  revised 
Instruction,  none  of  the  acquisition 
milestones  can  be  successfully  completed 
without  analysis  of  the  potential 
environmental  consequences  of  the  program 
and  development/identification  of  appropriate 
mitigation  measures.  DoD  Instruction 
5000.2  further  requires  that  the  Integrated 
Program  Summary  necessary  for  Milestones 
I,  II,  III,  and  IV,  contain  in  Appendix  E  the 
environmental  analysis.  Procedures  for 
conducting  this  analysis  are  contained  in  Part 
6,  Section  I  (4). 

The  problem  then  becomes  one  of  developing 
a  systems  acquisition  methodology  that  can 
incorporate  pollution  prevention  concepts  into 
it.  Fortunately,  the  basis  for  such  a 
methodology  already  exists  and  is  well 
established  in  the  acquisition  process  -  life 
cycle  costing.  Life  cycle  costing  has  been 
used  for  a  long  time  in  the  acquisition  process 
to  calculate  "cradle-to-grave"  costs  associated 
with  a  weapon  system.  This  concept  is  still 
valid,  except  that  "cradle-to-grave"  needs  to 
be  expanded  to  include  what  happens  when 
the  hardware  associated  with  a  weapons 
system  reaches  the  "grave".  Will  it  be  around 
for  hundreds  or  thousands  of  years  after  it  is 
disposed  of?  Will  its  residues  seep  in  to  local 
ground  water  supplies?  Will  the  cost  of 
ownership  continue  far  beyond  its  demise? 

Life  Cycle  Assessment  (LCA)  is  the  name 
given  to  the  current  approach  in  evaluating 
products  from  an  environmental  approach  - 
the  environmental  burden  of  a  weapon 
system.  An  LCA  evaluates  the  environmental 
releases  and  impacts  of  a 
product/process/activity  by  tracking  its 
development  from  raw  materials  through 
production,  and  ultimately  to  disposal.  LCAs 
are  not  a  new  phenomenon.  They  have  been 
used  for  approximately  the  past  twenty  years. 
Most  of  the  studies  have  been  funded  and 
conducted  by  private  firms  and  are  not 
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Figure  2.  Input-Output  Analysis  Framework 


generally  made  available  to  the  public. 

An  LCA  is  composed  of  three  phases.  Phase 
I  is  a  basic  input-output  inventory 
analysis.  Figure  2  presents  a  conceptual 


framework  for  conducting  this  analysis,  a 
process  which  can  be  applied  to  weapon 
system  acquisition.  A  very  important 
consideration  during  this  phase  is  the  amount 
of  energy  used  during  the  production, 
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operation,  support,  and  disposal  of  the 
weapon  system.  During  Phase  I,  data  are 
gathered  on  all  the  processes  and  intermediate 
products  that  go  into  the  production  of  a 
weapons  system.  These  data  are  gathered  for 
all  phases  of  the  acquisition  process.  This 
includes  data  on  the  raw  materials  used  in  the 
system,  manufacturing  processes,  operational 
procedures,  maintenance  requirements,  and 
ultimate  disposal. 

As  stated  previously,  most  LCA  studies  have 
been  conducted  by  private  firms  and  are  not 
available  to  the  public.  However,  a  few  are, 
and  they  have  gained  considerable  publicity. 
Among  these  are  a  1990s  study  by  Proctor  & 
Gamble  on  disposable  versus  cloth  diapers 
and  a  study  on  the  polystyrene  "foam" 
containers  used  by  McDonald's  Corp.  These 
studies  will  be  described  in  greater  detail 
later.  The  conduct  of  a  LCA  is  a  time 
consuming  and,  therefore,  expensive 
process.  This  helps  explain  why,  out  of  the 
millions  of  products  and  processes  in  the 
U.S.  economy,  only  a  handfull  of  studies 
(approximately  100)  has  been  performed. 

The  most  efficient  approach  to  performing  an 
LCA  is  to  utilize  existing  life-cycle  cost 
models  where  possible.  These  models 
contain  data  on  the  "nuts  and  bolts" 


of  the  components  and  subsystems  that  go 
into  a  weapon  system  and  can  form  the  basic 
level  of  an  LCA.  The  next  higher  level  in  the 
assessment  will  be  the  parametric  level  which 
will  quantify  system  performance  and 
introduce  risk  factors.  The  third  and  highest 
level  of  the  assessment  is  used  to  evaluate  the 
life-cycle  impact  on  the  total  environment  - 
the  environmental  burden.  Figure  3  presents 
a  conceptual  model  for  this  process. 

Phase  I  of  an  LCA  as  presented  above  is 
fairly  well  defined.  Depending  on  the  scope 
of  the  LCA,  additional  phases  might  be 
required.  Phase  II  deals  with  impact  analyses 
during  which  emission  quantities  and  their 
relative  environmental  consequences  are 
evaluated.  Whether  or  not  Phase  II  is 
performed  depends  on  the  objectives  of  the 
study.  If  an  impact  analysis  is  desired,  it  is 
imperative  that  what  constitutes  an  impact  in 
the  context  of  the  LCA  be  defined  initially. 
Typical  impact  definitions  range  from  human 
health  risks  to  habitat  alterations.  Phase  II  is 
not  well  defined  at  this  point  in  time  because 
the  relationship  between  an  environmental 
emission  identified  in  Phase  I  and  a  harmful 
effect  is  not  well  understood.  Furthermore, 
no  recognized  method  exists  for  comparing 
various  environmental  impacts.  For 
example,  if  the  input-output  analysis 
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Figure  3.  LCA  Analysis  Framework 


performed  in  Phase  I  reveals  that  a  specific 
production  process  will  generate  a  pound  of 
sludge  as  a  waste  product,  is  it  worth  the 
consumption  of  additional  BTUs  of  energy  to 
eliminate  it? 

Phase  III  deals  with  improvement  analyses. 
Certain  aspects  of  improvement  analyses  are 
also  not  well  defined.  These  typically 
revolve  around  making  value  judgments 
when  considering  whether  to  change  a 
process  or  activity  to  create  less  solid  waste 
but  perhaps  more  air  pollution.  This  type  of 
shortcoming  has  been  identified  and 
discussed  previously  when  describing  Phase 
II  activities.  However,  some  types  of 
improvements  can  be  identified  which  don't 
involve  a  decision  of  this  nature.  These 
improvements  are  implemented  and  result  in 
significant  cost  and/or  pollution  reduction  as 
shown  in  the  case  studies  described 
previously. 

While  providing  a  useful  tool  to  establish  the 
environmental  burden  of  a  specific  weapon 
system  which  is  produced  and  maintained 
using  specific  procedures  and  standards, 
LCAs  are  not  the  complete  answer.  The 
biggest  drawback  is  that  there  are  no  agreed 
upon  approaches  or  standards  to  use  when 
evaluating  a  system  being  acquired.  Value 
judgments  must  be  made.  For  example,  is  it 
better  to  use  more  BTUs  from  a 
nonrenewable  energy  source,  such  as  coal, 
possibly  increasing  air  pollution  during  the 
energy  generation  process,  to  produce  a 
product  which  will  be  more  biodegradable 
upon  its  final  disposal  (less  solid  waste)? 
Normative  questions  such  as  this  cannot  be 
answered  using  LCA. 

Lack  of  standards  can  also  produce 
inconsistent  results.  The  Proctor  &  Gamble 
and  McDonald's  studies  mentioned  earlier  are 
two  cases  in  point.  In  the  Proctor  &  Gamble 
study,  it  was  found  that  cloth  diapers 
consumed  more  than  three  times  as  much 
energy,  from  cradle  to  grave,  as  disposables. 
However,  a  study  sponsored  by  the  National 
Association  of  Diaper  Services  reported  that 
disposables  consumed  70%  more  energy  than 
cloth  diapers.  This  discrepancy  was  traced 
largely  to  accounting  methods  used  in 


measuring  energy  consumption  during  the 
processes  (1). 

In  the  McDonald's  study  performed  by 
environmental  consultant  Franklin  Associates 
Ltd.,  polystyrene  "foam"  containers  were 
shown  to  use  less  energy  in  the  production 
process,  pollute  less,  and  create  less  trash  bv 
weight  than  paper  containers.  However,  a 
study  by  environmentalists  concluded  that 
foam  is  not  biodegradable  and  makes  more 
trash  hv  volume  than  paper.  McDonald's 
eventually  phased  out  the  foam.  A  few 
months  later,  a  Canadian  chemistry  professor 
published  a  study  suggesting  that  foam  cups 
were  better  environmentally  than  the  paper 
ones  (5). 

LCAs  are  data  intensive  and  this  creates  a 
potential  problem.  It  is  difficult,  if  not 
impossible,  to  obtain  detailed  basic  data  for 
the  entire  life  cycle  of  the  product/activity 
being  studied.  Also,  some  data  may  not  be 
available  because  they  are  proprietary.  In 
these  types  of  circumstances,  the  use  of 
industrial  averages  obtained  either  from 
government  data  bases  or  trade  associations 
can  be  used  to  fill  data  voids.  Where 
possible,  sensitivity  analyses  should  be 
performed  to  determine  if  some  data  have  a 
disproportionate  impact  on  the  assessment. 
The  existence  of  such  data  would  necessitate 
greater  care  and  accuracy  when  obtaining 
them. 

Given  the  problems  mentioned  above,  of 
what  use  can  an  LCA  be  in  the  acquisition 
process?  There  are  three  primary  benefits 
which  can  accrue  from  the  completion  of  an 
LCA: 

•  LCAs  are  useful  in  identifying 
pollution  prevention  opportunities  in  various 
production  processes  used  in  acquiring 
weapon  systems. 

•  One  of  the  major  reasons  for 
conducting  an  LCA  is  to  compare  different 
approaches  to  the  development,  production, 
operation,  and  disposal  of  weapons  systems. 
Given  the  level  of  inaccuracy  in  determining 
the  absolute  level  of  environmental  burden 
produced  by  various  processes,  the 
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difference  between  them  is  useful  information 
in  deciding  which  is  the  best. 

•  The  DoD  is  a  major  consumer  in  the 
economy.  By  requiring  that  its  suppliers 
provide  some  type  of  LCA  for  the  products 
they  sell,  DoD  can  have  a  significant  impact 
on  the  total  burden  placed  on  our 
environment. 

CONCLUSIONS 

Until  recently,  one  of  the  basic  underlying 
assumptions  of  mankind  was  that  processing 
mankind's  waste  products  was  a  primary 
function  of  this  planet.  That  view  is  slowly 
changing  as  the  difficulties  of  disposing  of 
waste  material  continue  to  mount.  Achieving 
the  level  of  environmental  awareness  and 
sensitivity  necessary  to  ingrain  pollution 
prevention  into  the  acquisition  process  will 
not  be  an  easy  task.  Despite  its  shortcomings 
and  limitations,  LCA  provides  an  analytical 
tool  by  which  this  process  can  begin.  What 
is  needed  is  a  DoD  effort  to  conduct  an  LCA 
on  an  existing  weapon  system  in  order  to 
quantify  the  benefits  that  could  be  obtained 
using  this  methodology.  Existing  life  cycle 
cost  models  provide  a  possible  starting  point 
that  would  build  on  currently  available  data. 

Pollution  prevention  by  DoD  weapon 
systems  will  require  more  than  developing 
new  tools  and  changing  the  technology 
involved  in  the  production  process.  It  will 
require  changes  in  the  mindset  of  the  people 
who  develop,  produce,  and  purchase  weapon 
systems  for  the  DoD.  Perhaps  the  only  way 
to  achieve  such  a  mindset  change  is  to 
establish  a  national  commitment  similar  to 
that  which  helped  land  a  man  on  the  moon  26 
years  ago. 
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ABSTRACT 

This  paper  addresses  two  logical  consequences 
of  specifying  products  strictly  in  terms  of  their 
performance.  The  first  is  derived  from  conclu¬ 
sions  by  mathematical  logicians  that  certain 
types  of  problems  are  unsol vable  because  of 
their  great  computational  difficulty.  A  close 
correspondence  is  cited  between  one  such  type 
of  problem  and  the  task  of  evaluating  perfomr-- 
ance  specifications  for  engineering  feasibility. 
This  correspondence  accounts  for  the  great 
difficulty  that  engineers  seem  to  have  in  separ¬ 
ating  design  from  performance  in  specifications. 
The  second  logical  consequence  results  from 
practical  constraints  on  the  length  of  documents, 
and  a  simple  method  is  offered  for  avoiding 
the  difficulties  it  might  cause. 

INTRODUCTION 

The  customary  view  of  the  systems  engineering 
process  prescribes  that  designers  begin  their 
efforts  at  developing  a  new  product  by  first 
tabulating  and  validating  a  list  of  performance 
requirements.  The  validation  part  entails  a 
careful  check  to  be  certain  that  all  of  the 
requirements  are  essential  and  that  meeting 
them  is  economically  achievable.  In  Defense 
work,  our  usual  objective  for  the  past  several 
years  has  been  to  translate  the  list  of  validated 
requirements  into  a  performance  specification 
suitable  for  inclusion  in  a  Request  for  Proposals 
and  then  to  award  a  contract  for  the  completion 
of  the  product’s  design  and  development. 


However,  Government  engineers  and  scientists 
have  had  considerable  difficulty  confining  them¬ 
selves  to  specifying  requirements  based  only 
on  performance.  Despite  the  long-standing 
policy  that  dictates  the  use  of  performance 
specifications,  very  few  purely  performance 
specifications  have  been  produced.  Most  have 
consisted  of  a  mixture  of  performance  and  design 
requirements.  The  result  of  such  non-compliance 
has  been  the  establishment  of  a  formal  review 
process  to  ensure  that  the  policy  is  adhered 
to  in  the  future. 

Believing  in  the  premise  that  people  really  want 
to  do  their  jobs  correctly,  and  will  generally 
do  so  if  at  all  possible,  I  was  led  to  ask  what 
aspect  of  the  process  produces  such  a  disparity 
between  policy  and  practice.  My  study  of  the 
theoretical  aspects  of  specification  writing  has 
revealed  that  the  disparity  may  result  from  the 
nature  of  the  logical  relationship  between 
performance  and  design  and  from  the  difficulty 
involved  in  anticipating  the  design  solutions 
derivable  from  a  baseline  of  purely  performance 
requirements.  Such  difficulty  suffices  to  explain 
why  engineers  are  not  prone  to  base  their  rea¬ 
soning  on  a  set  of  numerous  performance  require¬ 
ments.  Perhaps  it  also  explains  why  technologi¬ 
cal  advancement  usually  proceeds  in  small  steps 
by  incremental  refinement  and  improvement 
of  existing  designs. 
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PERFORMANCE  SPECIFICATIONS  AS 
LOGIC  PUZZLES 

The  easiest  path  towards  understanding  why 
it  is  difficult  to  find  designs  that  satisfy  a  large 
set  of  performance  requirements  is  to  see  how 
closely  the  task  corresponds  to  the  process  of 
solving  certain  types  of  puzzles  encountered 
in  the  study  of  logic. 

Most  of  us  are  familiar  with  the  type  of  logic 
puzzle  that  consists  of  a  series  of  statements 
about  members  of  a  set  of  people  or  objects 
and  requires  that  one  reach  a  conclusion 
satisfying  all  of  the  given  constraints.  A  classic 
example  is  Lewis  Carroll’s  pork-chop  problem1, 
which  begins: 

(1)  A  logician,  who  eats  pork  chops 
for  supper,  will  probably  lose  money; 

(2)  A  gambler,  whose  appetite  is  not 
ravenous,  will  probably  lose  money; 

(3)  A  man  who  is  depressed,  having 
lost  money  and  being  likely  to  lose  more, 
always  rises  at  5  A.M.; 

(4)  A  man,  who  neither  gambles  nor 
eats  pork  chops  for  supper,  is  sure  to 
have  a  ravenous  appetite;  .  .  . 

The  eleven  additional  premises,  which  are  not 
necessary  to  illustrate  the  point  at  hand,  are 
given  in  the  endnotes. 

The  solution  to  the  pork-chop  problem  is  far 
from  obvious.  Even  the  number  of  people  in¬ 
volved  in  the  solution  is  not  evident  before  the 
problem  is  solved.  This  is  because  there  are 
eleven  variables  involved  and  fifteen  equations 
governing  their  interrelationships,  each  constrai¬ 
ning  either  three  or  four  variables.  Such 
solutions  are  beyond  the  capability  of  the  unaided 
human  mind,  which  Miller  has  shown  to  have 
a  short-term  memory  capacity  of  only  seven 


items,  plus  or  minus  two2.  Today's  approach 
to  solving  this  problem  would  be  to  write  a 
computer  program  that  automatically  formulates 
hypotheses  and  tests  them  for  compliance  with 
each  of  the  premises.  For  problems  of  this 
size,  the  necessary  exhaustive  search  can  be 
completed  by  computer  in  a  short  time,  but 
the  time  required  would  increase  very  rapidly 
if  additional  premises  were  added. 

By  the  way,  those  who  would  entertain  them¬ 
selves  by  attempting  to  solve  the  pork-chop 
problem  are  advised  to  review  the  literature 
first  because  some  of  the  semantics  and  punc¬ 
tuation  rules  have  changed  since  the  Victorian 
era  when  Lewis  Carroll  formulated  it1 .  Such 
seemingly  minute  details  as  the  semantics  of 
English  articles  and  rules  for  comma  usage 
are  also  extremely  important  to  specification 
writers3. 

The  parallel  between  such  logic  problems  and 
performance  specifications  can  be  seen  by  rec¬ 
ognizing  that  each  of  the  given  assertions  is 
a  statement  about  the  behavior  of  some  member 
of  the  group,  and  that  the  solutions  are  sets 
of  assertions  about  the  state  of  being  of  the 
members.  The  process  used  in  solving  the 
problem  identifies  sets  of  facts  that  constitute 
solutions.  This  is  essentially  the  same  process 
required  when  searching  for  designs  that  satisfy 
a  set  of  performance  requirements.  The  main 
recognizable  difference  in  form  between  the 
logic  problem  and  the  specification  is  that  the 
specification  requirements  are  written  entirely 
in  the  future  tense. 

As  stated  above,  the  complexity  of  the  problem 
depends  upon  both  the  total  number  of  variables 
and  the  number  of  variables  affected  by  each 
premise.  In  the  case  of  the  pork-chop  problem, 
these  are  relatively  few,  but  sufficient  to  make 
the  problem  too  difficult  for  an  unassisted  human 
mind.  In  the  case  of  the  specifications,  the 
number  of  statements  is  very  large.  My  research 
revealed  that  one  training  device  specification 
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stated  thousands  of  separate  requirements. 
The  interrelationships  between  statements  in 
the  specifications  are  not  always  explicitly  stated, 
but  they  exist  nonetheless  and  often  depend 
upon  decisions  made  in  the  course  of  design. 

In  addition  to  the  myriad  of  man-made  perfor¬ 
mance  requirements  that  must  be  satisfied,  there 
are  laws  of  nature  and  practical  constraints 
that  must  be  satisfied  as  well.  Every  designer 
knows  how  a  seemingly  minor  change  in 
requirements  often  produces  a  ripple  effect 
that  requires  the  readjustment  of  numerous  design 
details  throughout  an  entire  system.  For 
example,  many  specification  requirements  affect 
not  only  the  properties  they  explicitly  mention, 
but  also  affect  weight,  power  consumption, 
heat  dissipation  and  all  the  "-ilities. "  Likewise, 
every  performance  requirement  imposed  on 
software  has  an  impact  on  the  memory 
requirements  and  processor  speed  requirements. 
Practically  every  requirement  affects  cost,  which 
is  usually  constrained  by  the  figures  given  on 
the  front  page  of  the  contract. 

As  each  performance  requirement  is  added  to 
the  list,  it  has  to  be  evaluated  for  whether  or 
not  it  is  compatible  with  all  the  rest.  Many 
functions  each  have  to  work  both  independently 
and  with  all  combinations  of  certain  other 
members  of  the  set.  When  the  total  reaches 
twenty  of  such  compoundable  requirements, 
the  time  to  evaluate  them,  at  just  one  minute 
per  combination,  would  be  just  short  of  two 
years.  Thirty  such  requirements  would  create 
a  combinatorial  explosion  resulting  in  2043 
years'  work. 

While  it  is  evident  that  we  routinely  apply 
mathematics  to  solve  engineering  problems 
involving  large  numbers  of  variables,  one  must 
recognize  that  such  solutions  depend  upon  the 
representation  of  physical  phenomena  by 
operations  on  real  numbers,  and  continuity  of 
the  functions  involved  is  generally  essential 
to  the  solution  methods.  Such  methods  are 


not  applicable  to  problems  confined  to  the  logical 
domain  involved  in  the  process  of  evaluating 
a  product  for  acceptability.  A  product  either 
meets  each  of  the  performance  requirements 
specified  or  it  does  not.  If  it  were  to  meet 
all  but  one  requirement,  and  miss  that  one  by 
just  a  tiny  margin,  a  minor  revision  in  the 
product's  design  to  make  it  meet  that  one  re¬ 
quirement  might  very  well  cause  other 
performance  characteristics  to  fall  from  accept¬ 
ability.  This  is  particularly  true  of  the  kind 
of  highly  optimized  designs  that  are  characteristic 
of  Defense  work. 

SATISFIABILITY 

There  is  one  significant  difference  between 
the  logic  puzzle  and  the  design  problem:  We 
know  from  the  outset  that  at  least  one  solution 
to  the  logic  puzzle  exists.  Despite  our  best 
efforts  to  validate  the  performance  requirements, 
until  a  product  has  actually  been  produced  that 
satisfies  all  the  specification  requirements,  we 
are  never  really  sure  that  such  a  design  solution 
exists.  This,  again,  is  especially  true  in  Defense 
programs,  where  engineers  are  usually  striving 
to  advance  the  state  of  the  art. 

The  existence  of  a  solution  for  a  logic  problem 
is  called  "satisfiability."  It  is  clearly  a  pre¬ 
requisite  to  the  more  restrictive  requirement 
for  economic  feasibility  that  we  place  on 
engineering  specifications.  Economic  feasibility, 
in  turn,  is  a  prerequisite  to  the  legal  requirement 
of  "possibility  of  performance. "  Clearly,  one 
cannot  base  a  contract  on  a  set  of  requirements 
that  cannot  be  met. 

Modern  logic  theory  tells  us  that  determining 
whether  or  not  a  given  set  of  premises  is  sat- 
isfiable  belongs  to  a  class  of  problems  known 
as  "NP-complete,"  and,  by  our  analogy,  the 
satisfiability  of  most  every  large  set  of 
specification  requirements  is  also  likely  to  be 
NP-complete.  These  are  among  the  most  time- 
consuming  of  all  problems.  Satisfiability  for 
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problems  that  have  the  same  format  as  the  pork- 
chop  problem  but  contain  more  than  about  twenty 
premises  are  not  solvable  in  a  human  lifetime. 
For  problems  as  large  as  our  many-thousand 
requirement  training  device  specification,  the 
fastest  conceivable  computer  system  would 
take  more  space  and  time  than  the  universe 
has  to  offer1. 

While  it  is  not  unusual  to  discover,  well  into 
the  development  of  a  new  product,  that  some 
of  the  performance  requirements  will  have  to 
be  compromised  in  order  to  avoid  impossibility 
of  performance,  many  development  projects 
are  completed  smoothly.  How  can  this  be, 
considering  we  have  argued  quite  strongly  that 
determining  satisfiability  is  often  impossible? 

The  answer  is  that  the  systems  engineering 
steps  mentioned  above  are  not  really  repre¬ 
sentative  of  the  way  in  which  performance  re¬ 
quirements  are  derived.  Studies  of  artificial 
intelligence  have  revealed  that  the  human  mind 
has  evolved  methods  of  reasoning  that  avoid 
the  need  to  perform  exhaustive  searches.  In 
conceiving  a  new  product,  the  mind  seems  to 
approach  the  problem  in  the  reverse  direction. 
That  is,  the  newly  conceived  product  is  usually 
an  extrapolation  from  existing  designs  with 
which  the  systems  engineers  are  intimately 
familiar.  Their  performance-requirement  writing 
process  is  partly  like  that  of  the  logic-puzzle 
writer,  who  starts  out  with  the  solution  and 
writes  the  set  of  premises  to  fit  the  solution. 
For  this  reason,  design  is  necessarily  involved 
at  a  much  earlier  stage  than  what  our  process 
model  for  systems  engineering  postulates,  and 
the  necessity  is  a  consequence  of  the  manner 
in  which  all  automata,  including  the  human 
mind,  must  function. 

THE  "SHALL  NOT"  PROBLEM 

In  1975,  Puryear  warned  drafters  of  performance 
specifications  that  they  "must  activate  practically 
every  contingency  with  respect  to  the  functional 


requirements  of  the  product4, "  but  he  did  not 
elaborate  on  the  matter.  The  following  discus¬ 
sion  is  intended  to  illuminate  his  statement  and 
point  out  a  logical  flaw  in  our  present  approach 
to  drafting  performance  specifications. 

INCOMPLETENESS 

Consider  a  universe,  U,  of  performance 
characteristics.  U  is  a  set  that  contains  every 
possible  characteristic  of  every  possible  product. 
Of  course  there  is  no  such  thing  as  a  universal 
product  whose  performance  covers  all  of  U, 
but  every  product,  known  and  unknown,  could 
have  every  one  of  its  performance  characteristics 
mapped  onto  U.  In  Figure  1 ,  U  is  the  back¬ 
ground  on  which  the  diagram  is  drawn.  Simple 
reasoning  supports  the  assertion  that  U  contains 
a  countable  infinity  of  elements.  Design 
requirements  do  not  map  onto  U.  They  may 
produce  performance  characteristics,  which 
can,  in  turn,  be  mapped  onto  U,  but  design 
characteristics  themselves  are  not  in  U. 

Performance  requirements  can  be  mapped  on 
U.  They  are  merely  a  description  of  perfor¬ 
mance  characteristics  written  in  the  future  tense 
and  express  requirements  for  the  performance 
of  a  possibly  not-yet-existing  product.  A  set 
of  performance  requirements  is  usually  called 
a  performance  specification,  and  can  be  separated 
into  two  subsets:  S,  which  consists  of  those 
statements  that  express  requirements  that  the 
product  have  certain  performance  characteristics; 
and  S' ,  which  consists  of  statements  requiring 
that  the  product  not  have  certain  performance 
characteristics. 

In  a  formal  specification,  S  and  S'  would  be 
complete.  That  is,  they  would  totally  and  com¬ 
pletely  describe  everything  about  the  product. 
If  we  were  to  write  a  formal  performance  specifi¬ 
cation  and  add  the  statement  "The  product  shall 
do  nothing  else. , "  the  additional  statement  would 
have  no  effect  because  everything  necessary 
would  have  already  been  addressed.  Formal 
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specifications  therefore  usually  contain  a  very 
large  but  finite  number  of  requirements.  Formal 
specifications  must  also  be  written  in  formal 
languages  that  are,  of  necessity,  very  limited 
in  scope.  The  only  time  one  ordinarily  encoun¬ 
ters  a  formal  specification  is  in  the  code  of 
a  computer  program,  which,  by  the  way,  is 
a  formal  design  specification,  not  a  formal 
performance  specification. 

All  other  specifications  that  we  ordinarily 
encounter  are  informal.  In  informal  specifica¬ 
tions,  S  describes  performance  characteristics 
that  the  specifier  considers  important  enough 
to  mention,  and  S'  forbids  performance 
characteristics  that  the  specifier  considers 
important  enough  to  forbid.  Adding  the 
statement  "The  product  shall  do  nothing  else. " 
is  not  appropriate  in  such  cases,  and  would 
probably  constitute  breach  of  common  sense 
or  a  requirement  to  violate  some  law  of  nature. 

For  example,  if  the  color  of  the  specified  product 
were  not  mentioned,  the  only  way  the  product 
could  be  acceptable  under  a  "do-nothing-else" 
requirement  is  by  reflecting  no  light  at  all. 
The  only  things  known  to  come  even  close  to 
meeting  such  a  requirement  are  the  sky  on  a 
clear  night  and  holes  in  a  very  large  container 
whose  inside  has  been  made  as  nonreflective 
as  possible.  All  products  other  than  holes  and 
the  clear-night  sky  would  thus  be  excluded  from 


acceptability  by  the  "do-nothing-else" 
requirement. 

Given  the  necessity  that  performance  specifica¬ 
tions  have  to  be  open  ended,  it  is  therefore 
possible  that  a  product,  shown  as  P  in  Figure 
1,  may  have  some  performance  characteristics 
that  are  undesirable  but  not  forbidden  by  S' . 
Such  characteristics  might  make  the  product 
unsatisfactory  for  its  intended  use,  but  nonethe¬ 
less  acceptable  according  to  the  specification. 
More  often  than  not,  such  undesirable  character¬ 
istics  are  corrected  by  redesign  in  accordance 
with  an  engineering  change  proposal.  Occasion¬ 
ally,  such  a  characteristic  goes  unnoticed  until 
after  the  product  is  put  to  use. 

AN  ELEGANT  SOLUTION 

Clearly,  this  problem  will  not  be  solved  by 
attempting  to  forbid  all  possible  undesirable 
performance  characteristics,  since  their  number 
would  be  enormous.  A  good  alternative  would 
be  to  deviate  from  the  prescribed  outline  for 
specifications  and  move  the  usual  items  presented 
under  "Intended  use"  to  the  "Requirements" 
section.  Under  traditional  specification  outlines, 
"Intended use"  appears  under  "Notes,"  which 
does  not  contain  mandatory  requirements. 
Fitness  for  intended  use  is  a  perfectly  legitimate 
performance  requirement,  and  specifiers  are 
well  advised  to  always  make  it  a  requirement 
and  to  provide  a  detailed  description  of  its  basis. 

CONCLUSIONS 

In  addition  to  the  need  for  a  minor  revision 
to  the  standard  outline  for  specifications,  there 
are  two  conclusions  that  can  be  reached  from 
the  analysis  presented.  First,  without  risking 
impossibility  of  performance,  we  cannot  permit 
personnel  who  are  not  intimately  familiar  with 
the  design  of  similar  products  to  haphazardly 
choose  a  set  of  performance  requirements  for 
a  new  product  and  then  expect  a  contractor 
to  produce  a  design  that  satisfies  those  potentially 
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unsatisfiable  requirements.  Second,  the  key 
to  successful  development  of  new  products  may 
be  the  care  taken  in  limiting  the  number  of 
requirements  that  are  new  to  the  task  at  hand 
and  were  not  imposed  on  similar  products  that 
are  already  in  use.  By  doing  so,  the  design 
space  that  must  be  searched  for  solutions  could 
be  kept  within  the  human  scale.  Perhaps  a 
careful  audit  of  specifications  to  count  the  "new" 
requirements  would  permit  program  managers 
to  estimate  the  probability  of  later  encountering 
engineering  difficulties  and  thereby  asess  program 
risk. 
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END  NOTES 

The  remainder  of  the  pork-chop  problem: 

(5)  A  lively  man,  who  goes  to  bed  before  4 
A.M.,  had  better  take  to  cab-driving; 

(6)  A  man  with  a  ravenous  appetite,  who  has 
lost  money  and  does  not  rise  at  5  A.M. ,  always 
eats  pork  chops  for  supper; 


(7)  A  logician,  who  is  in  danger  of  losing  money, 
had  better  take  to  cab-driving; 

(8)  An  earnest  gambler,  who  is  depressed  though 
he  has  not  lost  money,  is  in  no  danger  of  losing 
any; 

(9)  A  man,  who  does  not  gamble, and  whose 
appetite  is  not  ravenous,  is  always  lively; 

(10)  A  lively  logician,  who  is  really  in  earnest, 
is  in  no  danger  of  losing  money; 

(11)  A  man  with  a  ravenous  appetite  has  no 
need  to  take  to  cab-driving,  if  he  is  really  in 
earnest. 

(12)  A  gambler,  who  is  depressed,  though  in 
no  danger  of  losing  money,  sits  up  till  4  A.M. ; 

(13)  A  man,  who  has  lost  money  and  does  not 
eat  pork  chops  for  supper,  had  better  take  to 
cab-driving,  unless  he  gets  up  at  5  A.M.; 

(14)  A  gambler,  who  goes  to  bed  before  4  A.M., 
need  not  take  to  cab-driving,  unless  he  has  a 
ravenous  appetite; 

(15)  A  man  with  a  ravenous  appetite,  who  is 
depressed  though  in  no  danger  of  losing  money, 
is  a  gambler. 

According  to  Poundstone,  a  solution  to  the  pork- 
chop  problem  is:  "An  earnest  logician  always 
gets  up  at  5  A.M.  and  sits  up  till  4  A.M." 
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Abstract 

The  broader  political  and  economic 
context  in  which  major  defense  contracts  are 
let  has  far  more  long-term  significance  for 
cost  and  for  the  preservation  of  important 
defense  production  capabilities  than  the 
micro-level  issues  of  specifications  and 
oversight.  Furthermore,  although  some 
detail-level  reforms  are  unquestionably 
desirable  for  their  benefits  in  streamlining  the 
acquisition  process,  other  *  announced 
directions  for  reform  fail  to  recognize 
important  contextual  constraints  and  may 
actually  compound  the  inherent  difficulties 
for  acquisition  posed  by  political  and 
technological  uncertainty.  Acquisition 
reform  initiatives  which  rely  on  market 
forces  to  impose  efficiency  will  be  thwarted 
by  politically-induced  market  failures,  and  an 
oversupply  of  major  production  lines  will 
maintain  high  overhead  cost  in  the  defense 
industry.  Moreover,  the  current  response  to 
rapid  technological  change  in  commercial 
products  and  to  even  faster  change  in  the 
political  winds  is  a  holdover  from  the  Cold 
War  habits  which  led  to  concurrency: 
initiatives  to  speed  acquisition  and  to  link  it 
to  "off-the-shelf'  technology  cannot  "catch 
up;"  they  simply  compound  projects' 
technological  uncertainty  and  lead  to  more 
unfulfilled  expectations.  A  more  sensible 
response  to  the  acquisition  crisis  would  slow 
the  entire  process. 

Introduction 

The  broader  political  and  economic 
context  in  which  major  defense  contracts  are 
awarded  has  far  more  long-term  significance 
for  weapons  system  costs  and  the 
preservation  of  important  defense  production 
capabilities  than  the  micro-level  issues  of 
contract  specifications  and  oversight. 
Furthermore,  although  some  detail-level 
reforms  of  the  weapons  acquisition  process 
are  unquestionably  desirable  for  their  benefits 
in  streamlining  the  acquisition  activities, 
other  announced  directions  for  reform  fail  to 
recognize  the  important  contextual 
constraints  and  may  actually  compound  the 
inherent  difficulties  for  acquisition  posed  by 
political  and  technological  uncertainty. 


During  the  Cold  War,  a  broad 
consensus  on  security  policy  was  forged  by 
the  perceived  threat  from  the  Soviet  Union. 
Acquisition  problems  were  rooted  in  a 
vicious  cycle  of  political  and  technological 
uncertainty  in  which  new  projects  over¬ 
promised  performance  improvements  based 
on  not-yet-developed  technology  to  satisfy 
the  multiple  political  masters  in  the  services, 
the  Department  of  Defense,  and  the 
Congress.  The  tendency  was  to  undertake 
concurrent  development  and  production 
commitments,  which  led  to  expensive 
programs  that  failed  to  satisfy  expectations. 
Because  of  the  general  agreement  on  the 
threat,  the  Cold  War  procurement  system 
was  largely  protected  from  use  as  a  political 
pork  barrel;  because  expectations  were  too 
high,  the  system  produced  the  best  military 
equipment  in  the  world,  even  in  "failure." 

With  the  Soviet  threat  now  absent, 
the  level  of  political  uncertainty  in  defense 
programs  has  soared:  there  is  no  longer  an 
agreed-upon  strategic  constraint  on  purely 
parochial  interests.  Large-scale  acquisition 
projects  are  generated  to  maintain 
concentrations  of  employment  and  incentives 
to  preserve  historically-important  corporate 
name  plates.  Acquisition  reform  initiatives 
which  rely  on  market  forces  to  impose 
efficiency  will  be  thwarted  by  politically- 
induced  market  failures,  and  an  oversupply 
of  major  production  lines  will  cause 
continuing  high  overhead  cost  in  the  defense 
industry.  Readiness  budgets  will  be 
squeezed  as  the  overall  defense  budget 
ceiling  lowers.  Moreover,  the  current 
response  to  rapid  technological  change  in 
commercial  products  and  to  even  faster 
change  in  the  political  winds  is  a  holdover 
from  the  Cold  War  habits  which  led  to 
concurrency:  initiatives  to  speed  acquisition 
and  to  link  it  to  "off-the-shelf'  technology 
cannot  "catch  up;"  they  simply  compound 
projects'  technological  uncertainty  and  lead 
to  more  unfulfilled  expectations. 

A  more  sensible  response  to  the 
acquisition  crisis  would  slow  the  entire 
process.  In  past  eras  of  low  threat,  defense 
production  capability  was  maintained  by 
public  arsenals.  Today,  the  equivalent 
producers  are  privately  owned,  but  the 
buying  relationship  should  mimic  the  old, 
lower-pressure  system.  At  presently- 
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envisioned,  low  rates  of  production, 
production  efficiency  is  a  less  important 
concern  than  wasteful  excess  capacity  and 
overhead. 

The  Dynamics  of  Cold  War  Defense 
Procurement 

To  analyze  the  problems  facing  the 
acquisition  policy  of  the  future,  it  is  useful  to 
set  up  a  framework  based  on  an 
understanding  of  the  procurement  problems 
of  the  past.  Today's  American  defense 
industry  was  created  by  the  pressures  of  the 
Cold  War,  as  were  the  present  government 
institutions  which  structure  its  sales  efforts. 
Many  of  the  core  Cold  War-era  issues  in  the 
development  of  advanced  weaponry  are  still 
with  us  today,  but  in  different  forms. 

Issues  Confronting  the  Cold  War 
Acquisition  Process 

The  fundamental  point  to  make  about 
the  Cold  War  is  that  the  United  States 
approached  its  international  and  defense 
policies  as  if  we  were  actually  at  war. 
Immediately  after  World  War  II,  defense 
production  was  radically  cut  back,  as  it  had 
been  after  World  War  I  and  all  other 
previous  American  wars  [1][31].  But  quite 
rapidly,  before  the  serious  impacts  of 
contraction  were  felt,  the  Soviet  threat 
emerged  and  returned  the  U.S.  defense 
budget  to  an  expansionary  path.  The 
January,  1948,  report  of  the  President's  Air 
Policy  Commission  (the  Finletter  Board) 
suggested  rapid  increase  in  aircraft 
purchasing  to  the  high  level  of  30-40  million 
airframe  pounds  annually  in  addition  to 
civilian  demand  to  provide  a  sound  basis  for 
emergency  production  expansion  [25]. 
Although  prevailing  American  economic 
ideology  (and  Truman's  attempts  to  hold  the 
budget  in  balance)  prevented  the  military 
budget  from  swamping  the  civilian  economy, 
a  substantially  higher  budget  share  for 
defense  procurement  than  in  any  previous  era 
of  "peace"  was  enshrined  in  the  Cold  War 
political  system  [7],  Cyclical  variations  in 
the  budget  continued,  but  the  broad  constant 
was  an  elevated  spending  level  [14]. 


Each  service  adopted  a  different 
strategy  for  spending  its  procurement 
budget,  but  all  involved  relatively  high-rate 
acquisition  of  advanced  weaponry  [4],  Very 
soon  after  World  War  II,  the  goal  of  defense 
production  began  to  shift  from  pure  mass 
production  to  innovation  and  research 
spending  [2],  Strategists  suggested  that 
America's  natural  comparative  advantage  in 
competition  with  the  Soviets  was  in  quality 
rather  than  quantity  [9],  Nevertheless, 
Congress  was  constantly  urged  to  build  large 
numbers  of  weapons.  "Small"  quantity 
production  was  quite  large:  1,923  B-47s 
were  build  during  the  program's  short  life 
from  1944  to  1957.  As  unit  costs  of  high- 
tech  aircraft  soared,  total  production 
dwindled  —  but  the  1 00-aircraft  run  of  the  B- 
1B  would  be  quite  large  by  today's 
standards.  Meanwhile,  new  technologies 
were  purchased  very  quickly,  often  before 
designs  had  been  fully  tested  and  all 
modifications  were  completed  [3],  The 
hurry-up  mentality  is  confirmed  by  the 
continued  use  of  concurrent  development 
strategies  even  after  it  had  become  quite 
clear  from  experience  that  it  often  led  to  cost 
over-runs,  technical  difficulties,  and 
extensive  retro-fitting  programs,  which 
called  into  question  the  strategy's  expected 
deployment  date  advantage  [3]  [3  2], 

All  weapons  development  programs, 
in  fact,  were  beset  with  two  types  of 
uncertainty  which  made  their  "success"  an 
unlikely  prospect.  One  uncertainty  derived 
from  the  nature  of  high-technology  weapons 
("technological  uncertainty");  the  second 
resulted  from  the  fragmentation  of  political 
authority  in  American  government  ("political 
uncertainty").  These  two  uncertainties 
interact,  and  in  fact  can  be  applied  to  all 
American  weapons  development  programs, 
including  those  of  today  [29], 

Technological  uncertainty  is  a 
product  of  the  process  of  invention  and 
expansion  of  knowledge.  In  the  quest  to 
ensure  that  the  United  States  possesses  the 
best  weapons,  the  military  constantly  pushes 
the  limits  of  what  is  possible  with  current 
technology.  Moreover,  even  as  we  attempt 
to  develop  affordable,  deployable  defenses, 
the  threat  becomes  more  capable,  and  the 
interaction  of  competing  technologies  adds 
to  uncertainty.  If  we  had  a  fixed  objective, 
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we  might  be  able  to  pursue  a  leisurely 
development  process  followed  by  an 
extensive  period  of  test  and  evaluation 
before  large-scale  production.  But  with  a 
shifting  threat,  sequential  development 
appears  impractical:  weapons  become 
obsolete  before  deployment.  We  simply 
cannot  know  whether  the  missions  set  for 
our  v/eapons  are  possible  in  many  cases. 

On  the  political  side,  it  is  a  basic, 
often-ignored  fact  that  the  government  of  the 
United  States  was  created  with  the  explicit 
intent  of  constraining  the  exercise  of  political 
power.  The  Founding  Fathers  created  a 
system  of  checks  and  balances  that  would  pit 
"interest  against  interest."  Most  broadly,  the 
interests  of  the  Executive  balance  those  of 
the  Congress,  but  the  fragmentation  of 
authority  also  exists  within  each  branch.  The 
Department  of  Defense  consists  of  many 
offices  and  agencies,  and  even  the  services 
are  divided  into  weapon  communities, 
committees,  commands,  and  offices.  Within 
a  service  branch,  one  community  may  be 
deeply  committed  to  a  weapon  or  capability 
while  another  may  be  indifferent  or  even 
hostile.  Events  constantly  shift  power 
amongst  these  factions.  In  the  legislature, 
the  House  and  Senate  have  different  bases  of 
representation,  terms,  and  interests.  The 
agglomeration  of  locally  elected 
representatives  has  an  obvious  local  focus, 
and,  within  each  house,  committees  struggle 
for  dominance.  Congressmen  worried  about 
the  next  election  are  often  more  interested  in 
claiming  credit  for  enacted  changes  than  in 
the  long-term  effects  of  those  changes  and 
abstract  notions  of  the  "national  interest" 
[18],  The  strategic  climate  of  the  Cold  War 
exercised  a  restraining  influence  on 
pernicious  Congressional  activity  [17], 
However,  with  the  disintegration  of  the 
Soviet  Union,  this  restraint  is  now  absent, 
and  there  is  a  new  degree  of  political 
uncertainty  [30], 

Cold  War  Procurement  Outcomes 

The  combined  effect  of  political  and 
technological  uncertainties  led  some  Cold 
War  weapons  programs  to  be  perceived  as 
successes,  while  others  were  seen  as  dismal 
failures.  One  analyst  recently  inventoried  the 


academic  literature  discussing  major  systems 
acquisitions  and  classified  53%  of  the 
nineteen  cases  as  successes  to  47%  failures 
[10]. 

Critics  of  the  Pentagon's  approach  to 
weapons  development,  emphasizing  the 
supposed  failures,  pointed  to  rising  costs  and 
shortened  production  runs  as  a  threat  to  the 
viability  of  America's  defense  industrial  base. 
Even  so,  Jacques  Gansler,  for  example, 
acknowledged  that  the  weapons  systems 
purchased  during  the  Cold  War  were 
"extremely  capable"  [8],  The  Office  of 
Technology  Assessment  reported  in  its  April, 
1989,  survey  of  the  defense  industrial  base 
that  "the  United  States  was  the  undisputed 
technology  leader  of  the  world.  U.S. 
military  equipment  was  meaningfully  and 
undeniably  more  sophisticated  than  that  of 
the  Soviet  Union,  and  our  allies  sought 
American  technology  for  their  own  defense 
efforts"  [33],  The  report  went  on  to 
question  the  future  of  that  supremacy  given 
the  possibility  that  Soviet  technology  would 
catch  up,  but  those  fears  have  since  been 
proven  unfounded  by  the  collapse  of  the 
Soviet  Union.  America's  absolute  level  of 
technological  strength  was  never  really  in 
question. 

Cold  War-era  Acquisition  Reform  Efforts 

Efforts  to  reform  the  weapon 
acquisition  process  are  nearly  constant, 
although  major  initiatives  have  been 
undertaken  erratically.  The  defense  budget 
was  a  major  proportion  of  total  government 
spending,  making  it  a  regular  political  target. 
Moreover,  many  procurement  programs 
were  perceived  as  failures  due  to  the 
unreasonably  high  standards  set  in  response 
to  the  inherent  political  and  technological 
uncertainties.  And  in  the  American  system, 
the  natural  response  to  a  perceived  failure  is 
an  attempted  reform. 

At  the  same  time,  the  vagaries  of  the 
international  situation  constantly  led  to 
changes  in  the  evaluation  of  America's 
security  and  consequently  to  new  calls  for 
better,  cheaper  weapons  —  which  often 
required  a  political  explanation  for  why  the 
previous  acquisition  system  had  failed  and 
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hence  a  new,  blue  ribbon  commission-led 
reform  effort. 

Acquisition  reform  as  a  political  tool 
led  to  considerable  variation  in  its  actual 
implementation.  In  circumstances  of  high 
international  tensions,  reforms  were 
proposed  but  generally  not  implemented. 
Defense  budgets  tended  to  be  high,  and  the 
trend  was  upwards.  Congress  and  the  public 
were  temporarily  willing  to  bear  high  costs  in 
order  to  ensure  the  security  of  the  United 
States.  Nonetheless,  criticism  of  weapon 
procurements  arose  because  of  these  high 
levels  of  expenditure,  because  real 
inefficiencies  unavoidably  occurred  during 
rapid  expansions,  and  because  criticism  of 
inefficient  procurement  is  a  "safe"  way  to 
criticize  high  levels  of  defense  expenditure. 
Even  hawlash  politicians  viewed  acquisition 
reform  as  protection  against  being  branded 
as  always  willing  to  give  the  military  a  free 
hand.  With  international  tensions  high, 
studies  were  commissioned,  but  they  served 
primarily  as  "smoke  screens"  to  defend  the 
level  of  military  expenditures  [12],  Even 
serious  reform  efforts  were  unlikely  to  be 
implemented  for  fear  of  disrupting  the 
buildup  and  undermining  national  security. 

The  contrast  is  to  the  brief  periods  of 
detente  during  the  Cold  War.  The  defense 
budget  generally  fell  with  the  tension  level, 
and  both  the  military  and  the  civilian  defense 
establishments  supported  efforts  to 
economize.  Formal  implementation  was 
therefore  more  likely,  because  the  reforms 
were  a  useful  political  tool  to  stave  off  larger 
program  cuts. 

The  McNamara  reforms  of  the  mid- 
1960s  are  an  example  of  implemented 
reforms  that  still  Tailed  to  improve  perceived- 
procurement  performance  [11].  The 
acceptance  of  the  bureaucracy  was  coerced, 
and  the  reforms  were  largely  put  into  place. 
The  Total  Package  Procurement  policy  failed 
simply  because  it  was  ill-conceived:  it  did  not 
take  account  of  the  technological  uncertainty 
always  present  in  the  weapons  development 
process.  Forcing  the  financial  and 
contracting  structure  to  treat  weapons 
development  as  if  systems  were 
technologically  mature  while  still  demanding 
prodigious  advances  to  meet  ambitious 
operational  goals  was  fated  for  disaster. 
McNamara's  reforms  illustrate  the  futility  of 


efforts  that  ignore  the  key  structural 
relationships  of  the  American  political  system 
and  the  associated  and  inter-related 
technological  uncertainty  characteristic  of 
our  acquisition  system. 

Current  Trends  in  Post-Cold  War 
Procurement  Policy 

In  the  post-Cold  War  period,  the 
context  in  which  procurement  institutions 
operate  has  radically  shifted,  but  the 
institutions  themselves  have  not  changed. 
The  abatement  of  the  Soviet  threat  has  left  a 
vacuum  where  there  used  to  be  international 
tension,  and  the  defense  budget  has  shifted 
downward  in  secular  rather  than  cyclical 
fashion  for  the  first  time  in  fifty  years.  The 
conditions  are  ripe  for  a  major  reform,  but 
the  present  effort,  by  relying  on  outdated 
concepts,  continues  and  even  exacerbates 
Cold  War  political  pathologies. 

Issues  Confronting  the  Acquisition  Process 

The  $252  billion  defense  budget 
approved  by  Congress  for  fiscal  1995  has 
brought  spending  just  below  the  1979  level 
in  real  dollar  terms,  signaling  the  end  of  the 
1980s  boom  cycle.  But  with  the  absence  of 
a  superpower  threat,  there  is  no  national 
security  reason  to  keep  the  budget  at  even 
this  level.  To  the  extent  that  the  defense 
budget  has  been  propped  up  in  the  1990s,  it 
has  been  to  protect  the  jobs  of  defense 
industry  workers,  covered  by  political 
rhetoric  about  the  industrial  base.  With  cuts 
only  now  threatening  the  defense  industry's 
core  capacity  rather  than  the  enlarged  size 
built  up  in  the  Reagan  years,  the  real  political 
action  is  just  about  to  begin  (even 
acknowledging  that  the  cuts  so  far  have  hit 
some  particular  sectors  harder  than  the 
industry-wide  average). 

The  FY95  authorization  bill  was 
chock  full  of  industrial  base  references,  from 
the  "bomber  industrial  base"  to  the  "ration 
industrial  base"  [28],  While  Newt  Gingrich, 
the  Speaker  of  the  new  Republican-led 
Congress,  rhetorically  supports  a  shift  in 
policy  from  old  idea  of  the  "industrial  base" 
to  the  Twenty-First  Century  notion  of  an 
"information  base,"  practical  GOP 
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implementation  proposals  favor  saving  big 
Pentagon  programs  such  as  the  B-2  bomber, 
missile  defense  systems,  and  the  Seawolf 
attack  submarine  -  protecting  concentrated 
interests  in  employment  [26], 

To  combat  the  perceived  defense 
industry  over-capacity,  the  government  has 
none  too  subtly  spread  the  word  that 
mergers  and  consolidations  will  be  looked 
upon  favorably  by  anti-trust  authorities.  The 
recent  approval  of  the  merger  between 
Lockheed  and  Martin  Marietta  has  been 
hailed  as  a  signal,  and  defense  analysts 
speculate  that  competitors  who  might  face 
"unfair"  competition  from  the  new  giant  have 
not  vigorously  opposed  the  deal  because 
they  fear  retaliation  from  the  Pentagon, 
which  strongly  supports  it  [23].  The  trouble 
with  the  current  approach  to  industry 
consolidation  that  "lets  the  market  decide" 
which  firms  will  survive  is  that  contractors, 
left  to  their  own  devices,  are  responding  to 
the  post-Cold  War  era  not  by  emphasizing 
efficiency  or  weapons'  capability,  but  instead 
by  lobbying.  Voluntary  corporate 

restructuring  is  painful:  share  prices  drop, 
and  workers  lose  their  jobs.  Shareholders 
and  workers  can  instead  beg  Congress  for  a 
restoration  of  their  budget  life-line 

Although  recent  mergers  appear  to 
be  a  kind  of  restructuring,  they  are  part  of  a 
lobbying  strategy.  Technical  "synergies" 
between  firms  are  often  exaggerated,  as  are 
manufacturing  plant  consolidations.  For 
example,  most  of  the  facilities  closed  after 
the  Northrop-Grumman-Vought  merger 
were  already  scheduled  for  closure  by  the 
separate  companies  [5],  Similarly, 
Lockheed,  primarily  an  aircraft  company, 
and  Martin  Marietta,  primarily  an  electronics 
and  space  company,  do  not  operate  in  many 
of  the  same  areas  of  defense  contracting. 
Opportunities  for  consolidating  post-merger 
production  are  minimal,  and  any  technical 
synergies  which  might  be  achieved  are 
blocked  by  an  FTC-imposed  "firewall" 
limiting  inter-division  communication  in  the 
interest  of  preserving  market-like 
competition.  The  genuine  logic  of  these 
Wall  Street  transactions  is  either  an  industry 
belief  in  the  political  clout  of  large 
organizations  (which  would  presumably  lead 
to  non-market,  non-strategic  acquisition 
outcomes)  or  an  industry  demonstration  of 


responsiveness  to  the  perceived  requirements 
of  their  government  customer  (also  in  an 
attempt  to  salvage  some  politically-induced 
contracting)  —  or  both  [20]. 

Post-Cold  War  Procurement  Outcomes 

At  present,  there  are  somewhere  in 
the  vicinity  of  eight  production  lines 
producing  or  capable  of  producing  military 
aircraft,  depending  how  one  counts. 
McDonnell  Douglas  has  two  lines,  Lockheed 
has  three,  Boeing  has  one,  Northrop 
Grumman  has  one,  and  some  would  also 
count  the  now-"cooling"  Northrop  Grumman 
facilities  which  used  to  make  F-14s  and  the 
Rockwell  plant  which  used  to  make  B-lBs. 
Two  years  ago,  it  appeared  that  Northrop's 
B-2  line  would  naturally  close,  trimming  the 
stable  of  going  production  plants,  but  it  has 
instead  been  the  first  prominent  instance  in 
which  job-oriented  political  incentives  have 
preserved  an  extra  production  line.  In  the 
FY95  budget,  the  Air  Force  spent  $150 
million  to  keep  the  line  "warm,"  if  not 
producing  new  aircraft;  now,  the  debate  has 
re-opened  for  additional  production  orders 
starting  in  FY96,  although  no  one  can  pose  a 
credible  military  scenario  in  which  American 
forces  would  be  called  upon  to  use  even  the 
twenty  B-2s  which  we  already  have 
purchased. 

The  primary  effect  of  this 
institutionalized  overcapacity  is  to  force  the 
procurement  portion  of  service  budgets  to 
unreasonably  high  levels.  Even  if  the 
acquisition  of  all  the  various  models  of 
advanced  weapons  could  be  justified  on 
strategic  grounds,  the  low  rate  of  production 
called  for  by  post-Cold  War  politics  bars 
efficient  production  at  each  of  the  individual 
plants  [16],  The  ratio  of  overhead  spending 
to  actual  operating  costs  of  defense  plants  is 
soaring  —  with  overhead  defined  not  in  terms 
of  corporate  headquarters  costs  and  the  costs 
of  "doing  business  with  the  government"  but 
in  terms  of  real  estate  costs,  energy  costs, 
artificially  large  workforces  not  taking 
advantage  of  scale  economies,  etc. 

The  underlying  irrationality  is  caused 
by  the  Cold  War  style  of  defense 
procurement.  Defense  procurement 

spending  comes  in  large  packages  only  — 
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enough  to  sustain  a  production  program  at 
each  shot,  including  all  the  overhead 
expenses.  Political  cover  for  Congressional 
efforts  to  save  jobs  comes  in  the  guise  of 
industrial  base  rhetoric,  which  seems  to 
require  actual  production;  what  was  lost 
during  the  Cold  War  was  the  ability  of  the 
American  procurement  system  to  support 
workers  and  design  capability  without  paying 
for  the  high-rate,  large-scale  production 
contracts  demanded  by  private  industry  —  an 
ability  inherent  in  arsenal-style  development 
and  production. 

Post-Cold  War  Acquisition  Reform  Efforts 

The  current  acquisition  reform  effort 
fits  well  into  the  framework  presented  in  the 
discussion  of  Cold  War  reforms  [15], 
Historical  experience  suggests  that 
recommendations  of  major  pro-reform 
reports  are  implemented  only  in  times  of 
relatively  low  international  tension,  and 
although  security  crises  in  Bosnia  and  North 
Korea  regularly  occupy  newspaper  front 
pages,  direct  threats  to  the  United  States 
today  are  small  compared  to  those  of  the 
Cold  War.  Just  as  proposals  were  rapidly 
implemented  during  the  detente  of  the  early 
1970s,  the  Federal  Acquisition  Streamlining 
Act  of  1994  adopted  many  suggestions  from 
the  Januaiy,  1993,  report  of  the  Acquisition 
Law  Advisory  Panel  (Section  800  Panel). 
The  announced  goals  were  to  give 
acquisition  corps  members  greater  flexibility 
in  procurement  decisions;  to  bring  new, 
traditionally-commercial  suppliers  into  the 
defense  business  and  thereby  increase  access 
to  cutting-edge  technology;  and  to  speed 
development  programs  by  cutting  through 
bureaucratic  red  tape.  The  mentality  is  such 
that  the  defense  industry  is  supporting  even 
faster  reform  than  the  new  law  offers 
through  "a  challenge  to  'pick  the  lowest 
hanging  fruit'  ...  [and]  to  adopt  industry's 
'best  business  and  production  practices'" 
[13],  The  difficulty  with  these  reforms,  in 
fact,  is  that  they  fit  too  well  into  the  Cold 
War  framework:  the  thinking  which 
motivated  them  is  stuck  in  the  old,  political 
paradigm  of  acquisition  reform  which  is 
inappropriate  for  the  post-Cold  War 
environment. 


The  effectiveness  of  the  reform  in 
meeting  its  stated  efficiency  goals  rests  on 
political  and  technological  uncertainty.  One 
rationale  in  Secretary  Perry's  statement  on 
the  need  to  integrate  defense  and  non¬ 
defense  industries  is  to  provide  the  military 
sector  with  faster  access  to  advanced 
commercial  technology  [24],  The  recent 
demonstration  of  the  military  benefits  of 
technological  superiority  in  Desert  Storm 
and  an  increasing  emphasis  on  high- 
technology  economic  competitiveness  in  the 
commercial  sector  make  capturing  some  of 
commercial  America's  success  for  the 
defense  sector  politically  important. 
Unfortunately,  in  major  military  programs 
with  long  lead  times  and  production  spread 
over  many  years  (because  of  high  unit  costs 
for  complex  systems),  the  pace  of 
technological  change  is  much  faster  than  the 
pace  of  contracting  in  even  a  simplified 
procurement  process.  The  best  possible 
streamlining  program  will  never  allow 
"technology  insertion"  from  the  commercial 
sector  into  military  programs  to  catch  up 
with  the  pace  of  innovations.  A  reform  that 
provides  the  military  greater  access  to 
changing  commercial  technology  will  only 
increase  the  technological  uncertainty 
inherent  in  the  procurement  process,  thereby 
threatening  the  programs'  success.  At  best, 
technological  goals  will  remain  modest  and 
the  new  access  to  the  commercial  market's 
technology  will  not  have  helped  the  military, 
but  the  benefits  of  certain  procurement 
regulations  (which,  it  should  be  remembered, 
were  not  established  capriciously)  will  have 
been  lost.  At  worst,  expectations  for  new 
military  technology  will  be  raised  too  high, 
leading  directly  to  technological  over-reach 
and  program  failure. 

At  the  same  time,  political 
uncertainty  has  radically  increased  in  the 
post-Cold  War  world.  The  political 
structures  of  the  federal  government  have 
not  changed,  and,  in  a  rare  instance  of  inter¬ 
branch  cooperation  in  defense  policy,  the 
recommendations  of  the  Congressionally- 
sponsored  Section  800  panel  and  of  the  Vice 
President's  National  Performance  Review  are 
in  basic  accord.  On  the  other  hand, 
obtaining  agreement  about  the  requirements 
of  future  weapons  systems  will  be 
increasingly  difficult  in  a  world  with  high 
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levels  of  political  disagreement  about 
America's  military  strategy.  There  is  no 
longer  an  obvious  superpower  adversary 
limiting  this  political  conflict,  as  there  was 
even  during  the  low-tension  era  of  detente. 
Integrating  the  commercial  and  military 
industrial  bases  is  likely  to  prove  impossible 
as  the  political  process  frequently  changes  its 
perceived  strategic  requirements,  reacting 
only  to  the  most  recent  "threat." 

Because  the  Section  800  panel 
adopted  a  managerial  approach,  using 
efficiency  as  its  sole  metric,  and  because  it 
neglected  to  consider  fundamental 
environmental  shifts  in  the  post-Cold  War 
world,  its  recommendations  are  unlikely  to 
be  effective.  Problems  will  arise  despite  the 
passage  of  the  implementation  vehicle  at  a 
time  of  low  international  tensions  and  despite 
the  unusual  legislative-executive  agreement 
on  the  goals  of  the  reform.  Those  goals  are 
simply  the  wrong  ones. 

A  Proposal  for  “Real"  Acquisition 

Reform 

The  1994  procurement  reform  law 
was  doubly  pernicious  from  the  perspective 
of  successful  defense  budget  cutting.  Not 
only  will  the  specific  provisions  of  the  law 
run  aground  on  the  Cold  War  nemeses  of 
political  and  technological  uncertainty,  but 
the  perception  that  the  recent  reforms  were 
on  the  right  track  further  diverts  the  policy 
process  from  the  true  path.  Calls  for  pro¬ 
efficiency  changes  in  manufacturing  strategy 
—  with  answers  expected  from  the  Air 
Force's  Lean  Aircraft  Initiative,  the  Agile 
Manufacturing  Enterprise  Forum,  and 
within-industry  moves  to  Total  Quality 
Management  and  its  equivalents  —  miss  the 
basic  observation  that  the  leanest  form  of 
manufacturing  is  the  production  of  no 
weapons  at  all.  Shaving  efficiency  losses  out 
of  production  budgets  matters  when 
production  runs  are  long,  not  when  a  large 
order  is  for  twenty  aircraft  spread  over  five 
years.  At  such  low-rate  production,  the 
focus  on  within-plant  efficiency  should  be 
minimal;  instead,  the  issue  at  hand  is  how 
many  plants  to  keep  open. 


An  Activist  Policy  Towards  Defense 
Company  Mergers 

The  elements  of  the  right  answer  for 
post-Cold  War  acquisition  reform  already 
exist  in  nascent  form.  Although  the 
Department  of  Defense  has  not  played  an 
active  role  in  pairing  up  defense  firms  for 
mergers,  there  is  a  working  merger  policy 
which  has  helped  to  eliminate  a  small  number 
of  duplicative  defense  facilities  once  two 
private  firms  have  decided  to  combine  their 
operations.  Martin  Marietta  was  reimbursed 
some  $80  million  by  the  government  for 
costs  incurred  in  moving  the  General 
Dynamics  Space  Division  facilities  which  it 
purchased  from  San  Diego  to  Denver; 
similarly,  Hughes  Missile  Systems  received 
between  $300  and  $400  million  in 
restructuring  cost  repayments  relating  to  the 
move  of  the  former  General  Dynamics' 
missile  operations  in  San  Diego  to  Hughes' 
Tucson,  Arizona,  operations  center  [34], 
The  announced  policy  is  to  pay  restructuring 
costs  when  it  can  be  shown  that  the 
prospective  savings  to  the  government  in 
terms  of  lower  overhead  costs  are  greater 
than  the  restructuring  charge  itself,  but 
because  the  program  uses  U.S.  government 
money  directly  to  close  plants  and  lay-off 
workers,  it  has  been  controversial  politically 
[6],  Unions  have  begun  to  protest  the 
practice,  and  Representatives  of  districts  in 
which  plants  have  been  closed  have  asked 
whether  all  restructuring  cost  payments 
should  go  to  corporations  and  shareholders 
rather  than  also  acknowledging  the 
adjustments  problems  faced  by  workers 
[19][21], 

The  issue  is  likely  to  come  to  a  head 
in  the  Hughes  case,  where  a  competing 
defense  firm  has  raised  the  issue  of 
competitive  fairness:  the  merger  policy 
payment  helped  make  Hughes  a  more 
efficient  producer  of  the  Tomahawk  missile, 
placing  McDonnell  Douglas  at  a  relative 
disadvantage  in  bidding  on  the  major 
contract  for  the  next  five  years  of 
production.  McDonnell  is  quite  right  that 
they  were  treated  unfairly,  and  although  the 
particular  contract  award  protest  is  still 
pending,  the  only  outcome  available  in  that 
forum  to  remedy  the  harm  to  McDonnell  is 
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to  re-divide  the  production  contract  among 
the  two  firms  —  maintaining  industry 
overcapacity,  costing  the  government 
additional  money,  and  sending  the  signal  to 
defense  industry  firms  that  efforts  to 
consolidate  facilities  will  not  be  rewarded.  A 
more  appropriate  repair  to  the  Tomahawk 
situation  would  be  a  conscious  policy 
decision  to  expand  the  current  merger  policy 
to  help  assuage  the  damage  to  the  losing 
competitors,  not  just  to  encourage 
restructuring  payments  to  acquired  firms 
whose  facilities  were  closed.  What 
McDonnell  needs  is  a  form  of  "severance 
pay"  for  services  rendered  during  the  Cold 
War,  an  exit  subsidy  to  help  them  leave  the 
missile  business. 

Overcoming  Political  Market  Failures 

Ironically  there  is  a  policy  to  pare  the 
capacity  that  the  government  itself  maintains 
for  weapon  development  and  production,  but 
not  one  yet  for  main  problem.  The  Base 
Realignment  and  Closure  (BRAC) 
Commission  has  in  several  rounds  removed 
some  of  the  government’s  capacity  when  it 
has  closed  or  merged  various  arsenals, 
shipyards,  repair  depots  and  military 
laboratories  in  order  to  eliminate  what  is 
usually  referred  to  as  the  Department  of 
Defense’s  burdensome  logistical  tail.  The 
procedures  for  realignment  and  closure  of 
the  facilities  is  intentionally  insulated  from 
politics.  Neither  the  Congress  nor  the 
President  can  protect  a  particular  facility  at 
the  behest  of  a  local  community  once  it  has 
been  designated  by  the  BRAC  as  excess. 
Instead,  the  entire  set  of  BRAC 
recommendations  must  be  accepted  or 
rejected  as  a  package. 

The  contractors  have  encouraged  the 
closure  of  the  arsenals  in  the  hope  of  gaining 
the  work  that  was  once  assigned  to  these 
facilities.  But  most  of  the  work  involved  in 
weapons  development  and  production  is 
already  in  the  hands  of  the  contractors. 
During  the  cold  War  there  was  a  definite 


shift  toward  privatization.  Relatively  few 
government  arsenals,  shipyards,  and  the  like, 
in  fact  survived  to  witness  the  collapse  of  the 
Soviet  empire.  Because  of  this,  the 
consolidation  that  the  current  closures  of 
arsenals  generates  will  do  little  to  solve  the 
overall  excess  capacity  problem  in  defense. 
The  BRAC  does  not  apply  to  private 
arsenals  system  that  we  have  created  during 
the  last  fifty  years  to  supplement  and  then 
replace  the  public  arsenal  system 

With  the  BRAC  process,  as  limited  as 
it  is,  already  flagging  because  of  political 
pressures,  it  is  difficult  to  see  the  route  to  the 
structural  changes  that  are  needed. 
Complicating  the  situation  is  a  politics  that 
has  the  parties  or  at  least  the  Congress  and 
the  President  competing  to  add  to  rather  than 
subtract  from  defense  spending  in  the  face  of 
mounting  federal  deficits.  The  belief  persists 
that  defense  allocations  will  soon  turn  up 
dramatically  to  meet  a  new  threat  or  at  least 
to  keep  the  various  production  lines  open 
and  host  communities  happy. 

But  if  the  turn  up  does  not 
materialize  because  a  major  foreign  challenge 
fails  to  appear  and  domestic  entitlements 
absorb  whatever  resources  taxpayers  are 
willing  to  provide  then  the  armed  services,  as 
custodians  of  national  security,  will  have  to 
muster  the  political  will  to  alter  the 
government’s  relations  with  its  private 
arsenals.  Instead  -of— multi-billion  dollar 
weapons  procurement  we  need  the  steady 
nurturance  of  core  weapon  technologies,  the 
maintenance  of  key  skills  rather  than  the  rush 
to  field  new  weapons.  The  hectic  activity  of 
the  Cold  War  has  to  be  supplanted  by  the 
quietude  of  a  watchful  waiting  that  mark 
years  of  peace.  America  needs  to  learn  to 
hone  talents  for  war  that  are  likely  not  to  be 
in  much  demand,  but  have  to  be  ready  when 
needed.  It’s  a  problem  we  know  exists  in 
nuclear  weapons  and  have  yet  to 
acknowledge  sufficiently  in  the  rest  of  our 
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military  capabilities.  Tinkering  with 
acquisition  reform  is  not  the  way  to  address 
it. 
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Acquisition  for  Experimental  or  Test  Purpose  (AETP)  and 
Acquisition  Reform  -  Two  Years  Later 


Patricia  L.  Krabacher,  CACM 
US  Air  Force 

ABSTRACT 

Acquisition  for  Experimental  or  Test 
Purposes  (AETP)  is  an  acquisition  reform 
initiative  which  addresses  the  inability  to 
acquire  hardware  and  limited  testing  services 
for  small  dollar,  one-time  test  or  experiment 
procurement.  Streamlining  efforts  originated 
at  Wright-Patterson  Air  Force  Base 
(WPAFB)  in  Nov.  1992,  to  allow  certain 
non-competitive  acquisitions.  AETP 
acquisitions  are  small,  one-time  tests  or 
experiments  when  data  is  needed  for 
program  decisions,  technology  insertion  for 
existing  items  or  products,  to  validate 
suggestions  or  ideas,  determine  commercial 
products’  acceptability  (or  compatibility  to 
our  systems/personnel),  etc.  While  the 
instant  AETP  acquisition  is  non-competitive, 
AETP  can  actually  enhance  long-range 
competition  for  large  programs  and  enable 
better  technology  insertion  or  requirements 
definition.  Local  policy  was  developed  at 
the  Aeronautical  Systems  Center  (ASC) 
WPAFB,  OH  based  on  the  statutory 
authority  then  existing  at  10  U.S.C.  2274 
and  9504.  Work  remains  to  establish 
implementing  language  in  the  Defense 
Federal  Acquisition  Regulation  (DFAR)  and 
lower  level  Federal  Acquisition  Regulation 
(FAR)  supplements  before  Contracting 
Officers  and  Program  Managers  can  easily 
use  the  existing  AETP  statutory  authority. 
This  paper  documents  the  background  and 
colorful  history  of  the  AETP  statutes,  an 
overview  of  the  field  inputs  which  support 
the  need  for  this  authority,  a  case  history  of 
the  F-16  Night  Vision  Goggles  (NVG) 


Theresa  M.  Squillacote,  J.D. 

OSD,  Acquisition  Reform 

program,  and  the  changes  necessary  to  the 
DFAR  and  Air  Force  FAR  (AFFAR). 
Acquisition  Reform  involves  all  of  us  and,  as 
the  AETP  story  reveals,  basic  process 
changes  are  difficult  and  complex. 

INTRODUCTION 

The  Acquisition  for  Experimental  or  Test 
Purposes  (AETP)  concept  first  received 
national  exposure  at  the  June  1993, 
Acquisition  Research  Symposium  sponsored 
by  the  Defense  Systems  Management 
College  (DSMC).  The  paper,  titled 
"Acquisition  for  Experimental  or  Test 

Purposes  (AETP)"  >  was  presented  at 
DSMC  and  published  in  the  conference 
proceedings.  The  paper's  thesis  was  simple: 
AETP  statutes  [then  existing]  at  10  U.S.C. 
2274  and  9504  offered  more  efficient 
utilization  of  dwindling  government  and 
contractor  resources,  could  save  taxpayers 
money,  would  allow  quicker  technology 
insertion,  give  greater  flexibility  to  conduct 
test  programs  for  commercially  available 
products  or  technology,  and  could  generate 
more  meaningful  future  competitions  as 
technologies  or  products  are  proven  to  be 
useful  to  DOD.  In  addition,  the  1993  AETP 
paper  covered  the  legislative  history,  the 
genesis/need  for  AETP  authority,  the  local 
policy  and  procedures  implemented  at  WP 
AFB,  potential  applications,  technology 
transfer,  lease  acquisitions,  and  issues 
remaining  before  successful  implementation 
could  actually  occur. 
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A  lot  has  happened  since  that  1993  paper 
was  written!  AETP  remains  a  fascinating 
story  of  acquisition  reform  initiated  at  the 
field  level  which  is  now  being  considered  for 
DOD  wide  implementation  by  the 
Acquisition  Reform  office.  This  paper 
discusses  the  AETP  initiative  which  provides 
authority  to  acquire  (or  lease)  hardware  and 
limited  testing  services  for  small  dollar,  one¬ 
time  test  or  experiment  without  competition. 
The  non-competitive  acquisition  is  a  radical 
process  change  from  the  schedule, 
manpower,  and  strategy  requirements  of  the 
Competition  in  Contracting  Act  (CICA). 
Discussion  and  real  world  example  will  show 
that,  while  the  instant  AETP  acquisition  does 
not  provide  for  full  &  open  competition, 
AETP  can  actually  enhance  long  term 
competitions  on  programs  and  enable  better 
technology  insertion  and/or  requirements 
definition. 

Whether  AETP  is  ever  fully  implemented 
across  the  DOD,  one  lesson  stands  clear- 
acquisition  reform  and  process  change  conies 
slowly!  The  entire  bureaucracy  must  change 
paradigms  and  behavior  before  acquisition 
changes  will  be  successfully  implemented. 
Additionally,  field  experience  shows  that 
reform  initiatives  reduced  to  "guidance  only" 
changes  do  not  enable  our  DOD  acquisition 
workforce  to  better  serve  the  taxpayers. 
Whether  Congressional,  DOD,  Service 
specific,  or  local  reform  all  initiatives  must 
be  carefully  thought  out  (with  inputs  from 
all  levels  affected)  and  implemented 
(adequate  time  to  rewrite  implementing 
regulations,  conduct  test  cases,  determine 
facilities  &  equipment  needs,  and  develop 
training  both  specific  and  professional )- 
otherwise  reform  initiatives  are  doomed  to 
fail. 


SUMMARY  OF  AETP  INITIATIVE 


Background.  Statutory  language  existed 
for  almost  70  years  at  10  U.S.C.  2274  and 

9504. 2  FIG  1  reflects  the  statutory  cites 
which  have  authorized  non-competitive 
acquisition  over  the  years.  The  10  U.S.C 
2274  and  9504  authority  survived  the  1984 
CICA  legislation  although  the  non¬ 
competitive  authority  was  rarely  used.  It 
appears  that  local  commands  were  aware  of 
the  AETP  statutory  authority  in  1986/87,  but 
may  have  deemed  it  too  politically  sensitive 
(given  the  sweeping  CICA  legislation)  to 
flow  AETP  down  to  the  field.  Little  use 
was  made  of  the  authority  until  WPAFB 
personnel  recognized  that  a  serious  gap 
exists  throughout  the  AF  in  our  ability  to 
acquire  hardware/software  for  small  dollar 
one-time  tests  or  experiments. 

ASC  Implementation  of  AETP.  Local 
AETP  policy  was  initiated  at  ASC,  in  late 
1990  to  implement  the  authority  of  10 
U.S.C.  2274  and  9504.  After  nearly  two 
years  of  development  and  coordination  an 
ASC  policy  letter,  incorporating  the  AETP 
authority  in  the  local  ASC  FAR  Supplement, 
was  issued  in  Nov.  1992.  No  other  FAR, 
DFAR,  Service  specific,  or  local 
implementing  language  is  known  to  exist  for 
the  AETP  statutes.  Use  of  the  local  AETP 
authority  was  complicated  by  several  factors, 
and  in  FY  93  only  one  ASC  acquisition 

successfully  applied  the  AETP  authority. 
Once  the  language  of  the  FY  94  Defense 
Authorization  bill  reached  WP  AFB  and  the 
much  narrower  language  of  2373  was 
publicized,  several  acquisitions  which  had 
contemplated  AETP  authority  were 
abandoned.  Further,  the  local  ASC  policy 
was  revised  in  June  1994  (to  be  in  concert 
with  the  re-codified  statute  at  2373)  to 
restrict  use  to  only  ordnance,  signal,  and 
chemical  warfare  supplies.  For  all  practical 
purposes,  AETP  authority  appeared 
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impotent  for  the  vast  majority  of  ASC 
programs  which  are  aeronautical  in  nature. 


Section  800  Report.  An  unexpected 
complication  arose  when  the  Section  800 
Panel  recommended  deletion  of  the  2274 
language  and  modification  to  the  9504 
language  only  two  months  after  ASC  issued 
the  local  policy  letter!  Based  on  the  Section 
800  Panel  recommendation,  the  FY  94 
Defense  Authorization  Bill  repealed  sections 
2274  and  9504  as  "obsolete,  redundant,  or 
otherwise  unnecessary",  but  reestablished 
narrower  language  at  10  U.S.C.  2373.  The 
repeal  and  re-codification  of  the  statutes  has 
further  inhibited  use  of  the  AETP  authority. 

Acquisition  Reform.  Fortunately,  however, 
in  mid-April  1994,  the  OSD  Acquisition 
Reform  Office,  OSD(AR),  contacted  ASC 
personnel  to  inquire  about  the  local  AETP 
policy.  Some  field  needs/issues  hastily 
identified  served  as  the  focus  of  a  brief 
meeting  between  OSD(AR)  and  SAF/AQ 
personnel  wherein  the  AETP  initiative  was 
discussed.  Off  line  discussions  between 
ASC  and  OSD(AR)  personnel  resulted  in  a 
request  on  April  18,  1994  for  a  detailed 
support  package  which  could  be  used  to  help 
determine  whether  the  AETP  initiative 
should  be  included  in  the  1994  Acquisition 
Reform  bill  then  in  markup/conference  in 
Congress.  Inputs  were  received  from  twelve 
different  organizations  (within  and  outside 
ASC)  and  these  were  included  verbatim  in 
the  package  which  was  sent  to  OSD  on  13 
May  94.  These  inputs  were  utilized  by  OSD 
(AR)  personnel  who  determined  that  the  new 
2373  language  should  be  broadened  to 
include  aeronautical  supplies  as  intended  in 
the  original  10  U.S.C.  2274  language.  Such 
language  was  added  to  the  FY  95  Defense 
Authorization  bill  in  Aug.  94  t0  make  the 


current  statute  coextensive  in  scope  with  the 
previously  existing  law.  (See  statutory 
language,  Figure  2). 

Need  for  AETP.  Nine  years  have  passed 
since  Congress  enacted  the  Competition  in 
Contracting  Act  (C1CA)  legislation,  and  field 
acquisition  personnel  were  tasked  to  ensure 
that  all  responsible  sources  are  permitted  to 
compete  on  non-exempted  acquisitions  over 
$25,000.  The  FAR  Part  6  establishes  the 
policies  and  procedures  to  promote  full  open 
competition  in  the  acquisition  process.  C1CA 
procedures  require  that  sealed  bids, 
competitive  proposals,  combination  of 
competitive  procedures,  broad  agency 
announcement,  or  GSA  multiple  award 
schedules  be  used  UNLESS  one  of  the  seven 
statutory  exemptions  can  be  justified  under 
10  U.S.C.  2304: 

(c)(1)  -  Only  One  Responsible  Source 
(c)(2)  -  Unusual  &  Compelling  Urgency 
(c)(3)  -Industrial  Mobilization  or. 
Engineering,  Developmental,  or  Research 
Capability 

(c)(4)  -  International  Agreement 
(c)(5)  -  Authorized  or  Required  by  Statute 
(c)(6)  -  National  Security 
(c)(7)  -  Public  Interest 

However,  no  authority  exists  which  permits 
one  time  testing  of:  commercial  products, 
new  or  emerging  technologies,  design 
improvements,  etc.  unless  ALL  responsible 
sources  are  permitted  to  compete.  In  actual 
practice,  it  is  often  found  that  on  small 
dollar,  one-time,  test/experiment  acquisition 
we  can’t  write  a  strong  enough  Justification 
and  Approval  (J&A)  to  sustain  an 
exemption.  It  would  be  a  rare  test  process 
that  could  be  justified  as  being  a  (c)(1)  and 
performed  by  only  one  source.  In  the  post¬ 
cold  war  era  a  (c)(2)  urgency  is  more 
difficult  to  substantiate.  Exemptions  (c)(3), 
(4)  and  (5)  are  rarely  used  (certainly  in 
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AETP  applications).  Exemption  (c)(6)  is 
occasionally  not  sustainable  even  if  classified 
data  is  involved,  and  these  authors  have 
never  seen  (c)(7)  used!  AETP  acquisitions 
do  not  require  a  J&A  exemption  in  the  ASC 
implementation  which  relies  on  a  written 
Determination  and  Finding  instead.  What  is 
needed  is  flexibility  to  go  with  a  single  or 
selected  sources  for  one-time  testing  not  a 
new  C1CA  exemption  since  the  acquisition 
or  lease  costs  are  usually  small  (between 
$50,000  and  $500,000)  and  the  test  duration 
is  usually  short  (from  a  few  weeks  to  six  or 
nine  months). 

Test  data  is  often  needed  to  make  program 
decisions,  resource  allocations,  refinements 
of  requirements  or  specifications,  validate 
compatibility  with  other  existing  and 
contemplated  systems,  or  to  evaluate  a 
suggested  improvement.  CICA  restrictions, 
while  ensuring  fair  and  equal  opportunity  to 
compete  for  federal  acquisitions,  also  may 
inhibit  DOD's  ability  to  obtain  timely  data  on 
commercial  products,  new  and  emerging 
technologies,  manufacturer's  design 
improvements  for  products  already  in  DOD 
inventory,  and  urgent  safety,  life 
maintenance,  and  regional  conflict  issues  (i.e. 
Desert  Storm).  Recent  audit  findings 
indicate  decision  makers  did  not  have  test 
results  available  prior  to  making  the 
production  decision  on  42%  of  modification 

programs. 

Field  Inputs  on  Need.  Acquisition 
professionals  are  stymied  by  the  acquisition 
bureaucracy  and  many  recognize  that 
acquisition  streamlining  for  one-time  test  is 
sorely  needed.  We  want  to  be  able  to 
acquire  needed  test  data,  identify  the  best 
hardware  for  our  fighting  troops,  and 
integrate  new  technologies  and  commercial 
products  into  our  aging  weapons  systems. 
Consider  these  words  spoken  by  the  program 


managers,  engineers,  and  contracting  officers 
who  execute  our  acquisitions: 

a.  "AETP  will  enable  us  to  exploit 
advancing  technology  in  the  commercial 
sector  and  quickly  test  it  for  application  to 
DOD  use.  It  will  reduce  the  time  and 
paperwork  required  for  bringing  a  good  idea 
into  common  practice.  It  will  be  endorsed 
even  by  the  contractors  who  often  become 
so  bogged  down  in  testing/experimentation 
stage  that  a  good  system  dies  before  it  can 
go  to  competitive  bid.  AETP  will  signal  the 
Administration's  intejit  to  truly  improve  the 
acquisition  process" 

b.  "During  a  Component  Technology 

Improvement  Workshop,  some 

manufacturers  complain[ed]  it  was  difficult, 

if  not  impossible,  to  introduce  design 

improvements  to  their  products  once  the 

configuration  is  delivered  to  DOD.  The 

approach  described  above  [AETP]  would 

benefit  both  industry  and  DOD,  and  will 

allow  the  manufacturers  to  become 

knowledgeable  of  DOD  requirements.  This 

has  the  potential  to  further  improve  off-the- 
8 

shelf  components"  . 

c.  "There  was  a  specific  case  where  the 
FACTS  office  attempted  to  lease  three  (3) 
communications  test  sets  (radios).  The 
procedures  outlined  in  the  local  (ASC) 
AETP  statutes  were  followed,  but  we  were 
unable  to  obtain  coordination  from  the  Staff 
Judge  Advocate's  office.  We  were  forced  to 
place  ourselves  at  the  contractor's  mercy  by 
borrowing  the  test  sets  in  exchange  for  the 
test  results" 

d  "We  are  once  again  taking  a  giant  step 
backward  with  the  repeal  of  the  AETP 
statutes.  The  (local)  implementation  of  these 
statutes  has  been  one  of  the  very  few 
significant  acquisition  reforms  bestowed  on 
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Contracting  Officers.  In  most  cases,  COs 
are  told  to  make  something  happen,  and 
make  it  happen  fast,  well,  and  cheaply. 
Usually,  we're  left  with  whatever  methods 
we  personally  have  --  risky  short  cuts, 
personal  charisma,  and  downright  pleading  -- 
to  get  the  job  done.  AETP  is  different  from 
most  so-called  reforms  in  that  it  actually 
changes  the  process  rather  than  examining 
the  number  of  steps  in  the  process  and  how 
each  small  step  can  be  accomplished  or 
possibly  even  deleted.  AETP  is  the  first 
reform  to  look  at  the  "big  picture"  and  give 

COs  the  flexibility  to  work  smarter"10 

e.  "Using  AETP  and  my  usual  streamlining 
methods,  I  was  able  to  shave  at  least  two 
months  off  my  lead-time,  awarding  it  within 
a  week  of  its  critical  date  and  thereby 
enabling  my  customer  to  meet  a  tight 

schedule" 

f.  "One  AETP  [modification]  was  projected 
to  cost  less  than  $130K  for  one  special  type 
penetrator  to  be  tested.  The  other  AETP 
[modification]  would  have  cost  about  $65K 
for  a  special  type  of  penetrator.  Now  [that 
AETP  statutes  have  been  repealed]  what  do 
I  do?  Unless  my  program  manager  can 
rewrite  his  requirements,  I  do  not  see  these 
actions  as  in  scope.  Nor  do  I  see  it  possible 
to  support  a  sole-source  justification  for 
either.  Neither  is  it  directly  a  matter  of 
National  Security.  Urgency?  We're  not  at 
war  officially,  so  how  can  I  justify  urgency. 
The  GAO  told  me  a  few  years  ago  that 
urgency  meant  "the  Russians  are  at  the  gate". 
Only  One  Source?  Well,  no.  Although  this 
particular  ordnance  reflects  only  a  tiny  sector 
of  the  Defense  community,  at  least  one  other 
contractor  could  probably  —  given  enough 
hand  holding  -  do  the  work.  So  now  1  have 
to  start  from  scratch  and  spend  months, 
dragging  both  Government  and  contractor 
employees  through  a  minuscule  source 


selection  for  actions  that  could  almost  be 
done  under  small  purchase  procedures?  No 
wonder  the  general  public  looks  ^  at 

Government  spending  with  such  disdain!" 

g.  "It  should  be  noted  that  many  test  and 
evaluation  efforts  do  not  involve  major 
systems  and  high  dollar  funding  (with  the 
resultant  visibility).  Many  more 

improvement  and  modernization  programs 
could  be  based  on  simple  solutions  if 
acquisition  authority  existed  to  acquire  items 
in  a  non-competitive  mode.  Without  AETP 
authority,  requirements  personnel  must 
"create"  a  situation  where  "Unusual  and 
Compelling  Urgency"  is  the  basis  for  not 
competing  or  attempt  to  justify  "Only  One 
Source"  with  poor  supporting  data  in  order 
to  procure  items  for  test  and  evaluation 
when  it  is  known  that  multiple  approaches 
exist  which  could  possibly  meet  AF  minimum 

requirements" 

h.  "During  the  past  two  years, 

modifications  have  become  more  difficult  to 
accomplish  due  to  approval  process  changes 
and  reduced  funding.  The  Competition  in 
Contracting  Act  often  adds  to  these 
frustrations  by  restricting  access  to  existing 
and  evolving  technology.  Both  time  and 
money  can  be  saved  with  the  effective  use  of 
the  AETP  authority  by  allowing  access  to 
existing  technology.  In  many  cases,  costly 
and  time-consuming  development 

procedures  can  be  eliminated  by  adapting 
existing  technology  to  new  requirements. 
Noncompetitive  testing  or  experiment 
authority  is  needed  to  stimulate  timely  and 
affordable  modifications  that  will  improve 
safety,  reliability  and  maintainability,  and 
operational  capabilities  of  our  aging  fleet  of 

weapon  systems." 
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"SPIN-OFF"  BENEFITS  OF  AETP 

Other  than  the  direct  benefits  of  allowing 
our  acquisition  professionals  to  acquire 
needed  test  or  experimentation  data  in  a 
timely,  resource  favorable,  and  more  cost 
effective  manner,  consider  these  other 
benefits  of  the  AETP  process. 

Enhanced  Competition  &  the  PRDA. 

Consider  the  paradigm  of  competition  that 
CICA  has  created  in  our  process. 
Competition  is  not  enhanced  simply  by 
allowing  everyone  to  spend  money  on 
writing  and  submitting  proposals  which  must 
be  evaluated  using  carefully  structured 
evaluation  criteria.  Look  at  the  Program 
Research  and  Development  Announcement 
(PRDA)  process,  actually  a  form  of  Broad 
Agency  Announcement  (BAA)15.  The 
PRDA  process  has  become  an  extremely 
popular  method  to  stimulate  creativity  in 
R&D  for  military  applications.  The  PRDA 
process  publishes  the  solicitation  in  the 
Commerce  Business  Daily  (CBD). 
Proposals  are  received  and  evaluated  for 
general  technical  merit,  but  against  top  level 
assessment  criteria.  A  technical  assessment 
is  made  and  the  government  may  award  any 
number  of  contracts,  entire  proposals  or 
parts  of  proposals.  Contracts  are  negotiated 
for  selected  technical  proposals.  Rarely  are 
PRDA  contract  awards  protested.  The 
PRDA  process  actually  enhances 
competition  for  later  Dem-Val  or  EMD 
programs  that  utilize  the  PRDA  developed 
technology.  PRDAs  are  designed  to 
maximize  competition,  primarily  at  the 
exploratory  development  stage. 

PRDAs  are  also  used  in  the  System  Program 
Office  (SPO).  The  Advanced  Strategic 
Tactical  and  Infrared  Expendables  (ASTE) 
program  is  one  such  ASC  program  which 
awarded  five  PRDA  contracts  in  1991,  and 


now  in  1995  is  preparing  to  enter  a  Full  & 
.  .  16 
Open  competition  for  EMD. 

AETP  Process.  Conceptually  similar  to  the 
PRDA  process,  the  AETP  process  deviates 
in  several  ways,  yet  enables  the  DOD  to 
maximize  corporate  participation  while 
maximizing  flexibility  to  acquire  limited  one¬ 
time  test  or  experiment.  AETP  acquisitions 
which  exceed  the  small  purchase  limitation 
must  be  synopsized  in  the  Commerce 

Business  Daily  (CBD).  The  synopsis 
builds  a  minimum  of  six  weeks  into  an 
acquisition  cycle  time,  since  two  weeks  are 
required  to  generate  and  publish  the  synopsis 
and  30  days  is  a  statutory  minimum  response 
time.  In  times  of  national  emergency  or  for 
other  urgent  acquisitions,  or,  for  small 
purchase  AETP  acquisitions  the  need  to 
synopsize  might  be  waived.  An  AETP 
synopsis  published  in  the  CBD  acknowledges 
the  intent  to  look  at  a  specified  area  for 
limited,  one-time  test  or  experiment  and 
defines  the  Statement  of  Need  for  the 
testing,  identification  of  the  item  or  process 
to  be  tested,  and  method  of 
test/experimentation  believed  necessary  to 
accomplish  the  effort.  Prior  to  publication  of 
a  synopsis  the  government  already  has 
identified  a  known  group  of  potential 
offerors.  However,  other  companies  are  free 
to  respond  and  enter  into  dialog  with  the 
requirements  personnel,  and  ultimately  to 
submit  a  proposal  that  would  be  fully 
considered  by  the  government.  The 
government  would  perform  a  top  level 
assessment  (not  as  detailed  as  in  a  PRDA) 
and  the  requirements  personnel  would  select 
the  intended  sources  that  would  require  the 
negotiation  of  awards.  However,  with  AETP 
the  test  parameters  have  been  generally 
defined  and  the  assessments  do  not  provide 
the  basis  for  a  competitive  evaluation.  The 
essential  tenants  of  AETP  are  that  1)  the 
government  has  an  inherent  right  to  obtain 
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test  data  2)  acquisition  personnel  are  trained 
to  determine  the  best  acquisition  strategy  for 
one-time  test/equipment;  and  3)  AETP 
acquisitions,  when  documented  and 
synopsized,  as  required,  are  not  protestable. 
A  typical  competitive  acquisition  rigidly 
defines  the  government  needs  by  part 
number,  in  a  Statement  of  Work  or  in  an 
System  Requirements  Document.  We  tend 
to  think  that  only  by  defining  requirements 
can  offerors  propose,  and  the  government 
competitively  evaluate  the  proposals.  AETP 
acquisitions  would  allow  the  government 
needed  flexibility  in  a  rapidly  changing 
technological  and  fiscal  environment  to 
support  EMD  &  production  program 
requirements  definition,  and  improvements 
to  inventory  items,  to  fill  the  "gaps"  in 
IOT&E,  safety,  and  urgently  needed  test 
data,  (or  even  the  evaluation  of  items 
produced  by  commercial  /industry  standards 
in  lieu  of  MIL-SPEC/STDS)!  The 
maximization  of  flexibility  to  chose  differing 
test  methods,  items  of  test,  etc.  means 
competition  may  not  exist  across  all 
proposals. 

F-16  NVG  CASE  STUDY 
The  F-16  System  Program  Office  (SPO)  had 
been  directed  by  a  user  through  a  Program 
Management  Directive  (PMD)  to  modify  200 
aircraft  with  Night  Vision  Imaging  System 
(NV1S)  which  would  allow  pilots  to  wear 
Night  Vision  Goggles  (NVG)  making  the  F- 
16  more  combat  effective  and  safer  to  fly  at 
night.  The  SPO  was  to  conduct  Tactics 
Development  and  Evaluation  (TD&E) 
testing  with  eight  F-16  aircraft  to  evaluate 
the  operational  capability  and  suitability  of 
NVG  to  provide  F-16  pilots  with  enhanced 
night  mission  effectiveness.  The  testing  had 
a  fourfold  objective:  1)  evaluate  the 

available  NVGs;  2)  determine  the 
capability/suitability  of  NVG  with  cockpit 
lighting  modifications;  3)  determine  the 


effectiveness  of  NVG  compatible  ground 
position  marking  devices;  and  4)  determine 
the  capability  of  NVGs  with  LANTIRN.  In 
late  1992  the  SPO  determined  that  the 
recently  issued  ASC  policy  on  AETP 
provided  the  most  economical  use  of  SPO 
manpower  and  the  optical  acquisition 
strategy  and  technical  payoff.  Based  on 
discussions  with  Night  Vision  Lighting 
(NVL)  industry  the  SPO  anticipated  three  or 
four  AETP  awards,  each  estimated  to  cost 
$100,000  or  less,  which  would  have  allowed 
the  NVL  industry  leaders  to  actually  modify 
F-16  cockpits  with  their  unique  lighting 
schemes.  The  Air  Force  then  would  have 
flown  the  modified  cockpits  for  3-4  months 
of  flight  testing  and  AF  pilots  would  evaluate 
the  effectiveness  of  the  different  NVL  on  the 
flight  instruments.  Later  the  F-16 
requirements  personnel  would  have 
generated  specifications  for  a  follow-on 
competitive  acquisition  based  on  the 
preferred  NVL  technologies  and  which 
specified  instrumentation  must  receive  NVL. 
The  F-16  Program  Manager  for  the  NVL 
program  stated,  “AETP  would  have  allowed 
us  to  do  the  following:  1)  Support  the  ACC 
testing  requirement  in  a  timely  manner;  2) 
Evaluate  four  different  approaches  for  NVL 
installation  on  the  F-16  to  determine  the 
most  cost  effective  modification;  3)  Generate 
specifications  for  follow-on  NVL 
procurements  based  on  the  best  approaches 
from  each  of  the  contractors  for  various 
“zones”  of  the  F-16  cockpit;  4)  Provide 
cost  estimates  for  submittal  into  the  F-16 
POM;  5)  Determine  the  cost,  schedule,  and 
technical  risks  of  a  follow-on  program;  and 
6)  Avoid  the  pass-through  costs  associated 
with  contracting  with  the  prime  (F-16) 
„  18 

contractor. 

The  F-16  was  prevented  from  utilizing  the 
AETP  authority  and  finally  went  through 
another  organization  to  utilize  an  existing 
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contract.  All  of  this  took  19  months,  more 
money  and  produced  less  technically 
desirable  results  because  the  industry  NVG 
houses  were  subcontractors  to  the  prime.  In 
March  94,  a  contractor  submitted  an 
unsolicited  proposal  to  modify  an  F-16 
cockpit  with  NVL  at  No  Cost  (which  the 
SPQ  had  to  reject  because  it  wasn’t 
considered  “innovative  or  unique”  enough). 
The  F-16  again  attempted  to  utilize  the 
AETP  authority  in  March  94  only  to  find 
that  the  1 994  Defense  Acquisition  bill  passed 
months  earlier  had  repealed  the  statutory 
authority.  The  F-16  finally  abandoned  the 
effort,  but  their  experience  in  frustration  is 
not  unique  in  the  DOD. 

Contracting  Directly.  One  last  thought  on 
how  AETP  could  improve  the  perceived 
competition  we  currently  foster.  Award  of  a 
task  or  delivery  order  or  contract 
modification  for  a  needed  test  or  experiment 
may  be  possible  within  scope  of  a 
"competed"  contract.  However,  such  awards 
do  nothing  to  enhance  competition  and  are 
likely  to  cost  more  (than  going  direct  to  the 
technology  house);  produce  a  less  optimal 
technical  result,  and  take  longer  because  of 
added  layers  of  management.  The  AETP 
process  allows  the  government  to  contract 
directly  with  industry  for  needed  testing.  It 
enables  interested  companies  to  dialog  with 
the  requirements  personnel  who  are  free  to 
select  those  firms  who  offer  unique  solutions 
or  test  capabilities.  It  also  probably  means 
we  can  rely  less  on  existing  contract  vehicles 
(i.e.  task  order  or  use  of  the  “Changes” 
clause)  to  acquire  testing. 

Reduced  Manpower  Across  DOD.  As  the 

draw  down  continues  those  remaining  in 
acquisition  know  that  our  jobs  have  gotten 
harder.  We  have  fewer  personnel  to 
accomplish  a  mission  that  involves  more 
"false  starts",  funding  delays,  and  what 


seems  like  more  "political"  impacts.  We 
simply  don't  have  the  personnel  to  staff 
source  selections.  The  "streamlined" 
Appendix  BB  of  the  AFFARS  requires  a 
Source  Selection  Authority  (SSA)  and 
Source  Selection  Evaluation  Team  for 
acquisitions  over  $5  million!  An  Appendix 
BB  source  selection  typically  uses  30-35 
personnel  working  9-12  months!  An  AETP 
acquisition  (contractually  less  than  $5M) 
could  be  accomplished  with  4  or  5  personnel 
over  4  or  5  months. 

Acquisition  Reform.  AETP  authority 
conceivably  could  be  used  to  "test"  the 
implementation  of  Acquisition  Reform  across 
the  DOD!  Imagine  a  one-time  test  of  several 
EDI  systems  that  could  meet  the 
requirements  of  FACNET.  Now  that's  using 
your  AETP! 

Technology  Insertion. 

AETP  authority  can  be  used  to  test  the 
insertion  of  new  technology  into  existing 
systems.  This  function  is  especially 
important  where  we  are  seeking  to  upgrade 
existing  systems  rather  than  develop  new 
ones. 


REMAINING  EFFORT  TO  FULLY 
IMPLEMENT  AETP 

DFAR  Guidance  Needed.  In  Dec.  1994, 
the  Acquisition  Reform  Office  determined 
that  the  AETP  Authority  should  be 
implemented  DOD-wide.  This  decision 
resulted  largely  from  the  ASC  inputs 
submitted  on  13  May  1994.  The  DFAR 
language  will  be  modeled  after  the  ASC 
language  and  is  likely  to  include  guidance  at 
Part  206  to  clarify  the  non-competitive 
authority  and  at  Part  217  to  establish  the 
special  contracting  methods  used  for  AETP 
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acquisition.  Significant  work  remains  to 
develop  the  DFAR  guidance  and  OSD(AR) 
intends  to  work  to  rapidly  consider  the  four 
or  five  major  implementation  issues.  We 
welcome  your  inputs  on  this  implementation 
of  AETP  in  the  DFAR.  It  is  anticipated  that 
DFAR  language  will  be  available  for  public 
comment  late- 1995. 

Command  and  Center  Implementation. 

Once  the  DFAR  and  AFFAR  language  is 
implemented,  it  is  important  that  Command 
FAR  Supplements  flow  down  the  available 
statutory  authority,  otherwise  Contracting 
Officers  and  Program  Managers  will  not  be 
empowered  to  actually  use  the  authority. 

CONCLUSION 

Something  must  be  done  to  allow  the  DOD 
greater  flexibility  to  acquire  the  test  or 
experimental  data  required  to  make  the  best 
decisions  on  our  programs  and  acquisitions. 
One  solution  is  to  implement  AETP  language 
in  the  DFARS  to  ensure  contracting  officers 
and  program  managers  have  greater 
flexibility  to  conduct  one  time  test  or 
experiment  to:  a)  conduct  testing  of 
commercial  products  or  supplies  to  avoid 
unnecessary  development  costs,  b)  remove 
the  hindrances  (schedule,  technical,  and 
business)  experienced  under  the  Competition 
in  Contracting  Act  (CICA),  and  c)  foster 
more  efficient  use  of  acquisition  manpower 
and  taxpayer  funds.  Because  of  the 
procurement  lead  times  and  manpower 
restrictions,  CICA  requirements  and  other 
restrictions  we  force  acquisition 
professionals  to  1)  modify  requirements  to 
accommodate  available  business  strategy,  2) 
small  purchases  or  imprest  fund/credit  cards 
are  seen  as  the  only  avenue  to  acquire 
expeditious  testing,  3)  large  support  task  and 
delivery  ordering  contracts  are  used  routinely 
to  acquire  needed  testing,  4)  scope  is 


stretched  on  existing  contracts  to  acquire 
data  produced  by  testing,  or  5)  needed 
testing  goes  undone.  Reductions  in 
acquisition  personnel  and  budget  inability  to 
insert  new/emerging  technologies, 
empowerment  of  acquisition  personnel,  and 
even  common  sense  tell  us  that  we  must 
facilitate  the  application  of  10  U.S.C.  2373 
authority  to  preserve  the  ability  to  conduct 
limited,  one-time  test/experiment  when  it  is 
determined  necessary. 

Two  years  ago  we  concluded  that  the  AETP 
statutes  at  10  U.S.C.  2274  and  9504  held 
great  promise  of  savings.  In  the  name  of 
acquisition  streamlining  these  statutes  were 
repealed  and  revised,  respectively,  in  the 
1994  DOD  Defense  Authorization  bill!  Due 
to  quick  work  by  the  DOD  Acquisition 
Reform  office  the  1995  Defense 
Authorization  bill  expanded  the  newly 
established  authority  at  10  U.S.C.  2373  to 
include  aeronautical  supplies.  Will  the 
system  allow  a  paradigm  shift  on  how  we 
look  at  competition?  Are  the  DOD  policy 
makers  in  touch  with  the  reality  or  of  the 
field?  The  future  of  AETP  will  tell. 

Pat  and  Terry  would  like  to  thank  the 
contracting  officers  and  program  managers 
who  provided  inputs  and  especially  Lorna 
Tedder,  one  gutsy  and  smart  CO!  This 
paper  reflects  the  views  of  the  authors  and 
does  not  reflect  the  opinions  of  ASC  or  OSD 
(AR)  as  a  whole.  Contact  Pat  Krabacher 
ASC/LN  at  DSN  785-2425,  or  (513)255- 
2425,  ext  3709  or  FAX  476-4598  for 
additional  information.  Contact  Terry 
Squillacote,  OSD(AR)  at  (703)  614-3882  or 
Fax (703  )  614-1690. 
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2274.  Procurement  for  experimental  purposes. 

(a)  The  Secretary  of  a  military  department  may  buy  designs,  aircraft,  aircraft  parts,  and 
aeronautical  accessories  that  he  considers  necessary  for  experimental  purposes  in  the  development 
of  the  best  kinds  of  those  items  for  the  Army,  Navy,  Air  Force,  or  Marine  Corps,  as  the  case  may 
be.  Purchases  under  this  subsection  may  be  made  abroad  or  in  the  United  States  or  the 
Territories,  Commonwealths,  and  possessions,  with  or  without  competition  and  by  contract  or 
otherwise. 

(b)  If,  as  a  result  of  a  purchase  under  subsection  (a),  a  new  or  suitable  design  considered  to  be 
the  best  kind  for  the  Army,  Navy,  Air  Force,  or  Marine  Corps,  as  the  case  may  be,  is  developed, 
the  Secretary  of  the  military  department  concerned  may  contract  for  procurement  of  the  item  in 
quantity.  Contracts  under  this  subsection  are  subject  to  sections  2272(f)  and  2279  of  this  title  but 
are  not  subject  to  sections  227 1  (a)-(cl)  and  2272(a)  of  this  title. 

(Aug  10,  1956,  c.  1041,  70A  Stat.  126.) _ 


9504.  Procurement  for  experimental  purposes 

The  Secretary  of  the  Air  Force  may  buy  ordnance,  signal,  and  chemical  warfare  supplies, 
including  parts  and  accessories,  and  designs  thereof,  that  he  considers  necessary  for  experimental 
or  test  purposes  in  the  development  of  the  best  supplies  that  are  needed  for  the  national  defense. 
Purchases  under  this  section  may  be  made  inside  or  outside  the  United  States,  with  or  without 
competitive  bidding,  and  by  contract  or  otherwise.  Chapter  137  of  this  title  applies  when  such 
purchases  are  made  in  quantity.  Aug.  10,  1956,  c.  1041,  70A  Stat.  575. 


PUBLIC  LAW  103-160,  SECTION  822  10  U.S.C.  2373 

(a)  AUTHORITY.  The  Secretary  of  Defense  and  the  Secretaries  of  the  military  departments 
may  each  buy  ordnance,  signal,  chemical  activity  and  aeronautical  supplies,  including  parts  and 
accessories,  and  designs  thereof,  that  the  Secretary  of  Defense  or  the  Secretary  concerned 
considers  necessary  for  experimental  or  test  purposes  in  the  development  of  the  best  supplies  that 
are  needed  for  the  national  defense. 

(b)  PROCEDURES.  Purchases  under  this  section  may  be  made  inside  or  outside  the  United 
States  and  by  contract  or  otherwise.  Chapter  137  of  this  title  applies  when  such  purchases  are 
made  in  quantity. 
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Forest  Service 


ABSTRACT 

In  the  Spring  of  1993,  President  Clinton  held 
a  conference  in  Portland,  Oregon  in  an  attempt 
to  resolve  the  disputes  and  tension  between 
the  timber  industry,  environmental  groups, 
and  concerned  citizens.  The  focal  point  of  the 
discussion  involved  the  habitat  for  the 
Northern  Spotted  Owl.  The  larger  issue  was 
the  use  of  public  lands,  particularly  those 
remaining  that  were  largely  undeveloped  and 
still  covered  with  old  growth  timber.  Of 
particular  concern  was  the  impact  on  logging 
dependent  communities  as  National  Forest 
holdings  became  increasingly  unavailable  for 
harvesting  timber.  These  issues  have  long 
been  politically  sensitive  and  disputes  of  this 
nature  have  existed  as  long  as  the  Federal 
Government  has  been  involved  in  land  use 
policy.  However,  the  drastic  change  taking 
place  was  eliminating  the  jobs  of  many 
people. 

The  1994  Appropriations  Act  for  the  Forest 
Service  [1]  contained  funding  for  watershed 
improvement  in  the  Pacific  Northwest.  The 
impacted  areas  were  forested  areas  in 
Northern  California,  and  the  states  of  Oregon 
and  Washington.  Although  nothing  in  the 
legislation  addressed  procurement  law,  the 
Conference  Report  discussed  intent  to  spend 
money  for  Community  Assistance.  To  meet 
this  intent,  Secretary  of  Agriculture  Mike 
Espy,  signed  a  Determination  and  Finding 
invoking  exception  7  to  the  Competition  in 
Contracting  Act  (41  USC  253).  This 
essentially  freed  the  Forest  Service  from 
Competition  in  Contracting  Act  constraints. 


This  paper  describes  the  rules  developed  for 
selecting  competing  firms  and  elaborates  on 
how  this  program  functioned  during  its  first 
year  of  application.  Primary  issues  involved 
included  how  to  develop  a  program  that 
limited  contracts  to  firms  located  in  specific 
geographic  areas,  whether  there  should  be 
attempts  to  limit  workers  to  residents  of  those 
communities,  how  to  accomplish  outreach 
within  the  restricted  area,  and  how  to  provide 
assistance  to  new  firms  established  by 
displaced  workers.  The  basic  problem  was 
how  to  put  a  program  in  place  that  in  many 
ways  was  foreign  to  fundamental  ideas  in 
Government  procurement.  This  paper  will 
also  discuss  some  long  term  ramifications  of 
approaches  such  as  aiding  communities 
affected  by  political  actions  set  in  motion  by 
the  changing  desires  of  the  American  Public. 

INTRODUCTION 

The  United  States  Forest  Service  was 
established  early  this  century  as  a  major  land 
manager  for  the  United  States  Government. 
The  Forest  Service  manages  191  million  acres 
of  forests  and  grasslands.  The  Bureau  of  Land 
Management  manages  a  huge  portion  of  the 
United  States  range  land.  In  places,  the  land 
types  are  difficult  to  distinguish  but  a  good 
rule  of  thumb  is  that  the  Forest  Service 
manages  forested  areas  and  the  Bureau  of 
Land  Management  manages  areas  of 
rangeland.  The  National  Park  Service,  of 
course,  manages  the  national  parks.  Although 
the  areas  that  the  Park  Service  manages  are 
quite  diverse  (the  Washington  Mall  to 
Yellowstone  and  Glacier  National  Parks)  their 


Please  note  that  the  views  expressed  in  this  paper  are  those  of  the  author 
and  do  not  necessarily  represent  the  views  of  the  Forest  Service  or  the  Department  of Agriculture. 
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role  is  much  clearer.  They  have  the  role  of 
preserving  national  and  historic  treasures  and 
providing  recreation  for  the  American  Public. 
(They  might  define  it  differently  but  you  get 
the  point.)  On  the  other  hand,  the  Forest 
Service  and  the  Bureau  of  Land  Management 
have  always  had  the  competing  concerns  of 
both  using  the  resources  on  the  land  they 
manage  while  preserving  and  protecting  the 
resource.  The  Bureau  of  Land  Management 
has  at  least  two  areas  of  critical  extractive  use 
management  in  grazing  and  mining.  This 
paper  is  not  about  that  either.  The  land 
management  agencies  have  always  been  torn 
by  two  competing  interests  that  tend  to  be 
mutually  exclusive.  In  the  early  part  of  this 
century,  John  Muir,  who  believed  that  the 
public  lands  should  be  kept  as  close  as 
possible  to  their  pristine  state  for  the 
enjoyment  of  citizens  who  wanted  an  outdoor 
experience,  represented  one  side  of  the 
equation.  There  is  a  great  deal  of  written 
material  concerning  the  benefit  that  people 
derive  from  "having"  a  wilderness  experience 
or  simply  knowing  that  it  is  there  to 
experience  if  one  wished  to  do  so. 

On  the  other  hand,  Theodore  Roosevelt  and 
Gifford  Pinchot,  the  first  Chief  of  the  Forest 
Service,  believed  in  the  multiple  use 
philosophy.  They  believed  that  it  is  possible 
to  extract  resources  in  a  careful  way  and  still 
preserve  recreational  and  other  somewhat  non¬ 
extractive  benefits.  This  position  is 
summarized  in  the  short  statement  of 
"Multiple  Use,  Sustained  Yield." 

The  tension  between  these  two  points  of  view 
has  never  been  resolved.  This  can  be  seen 
through  the  continuing  tension  between  the 
timber,  mining,  ranching,  fishing,  and  other 
extractive  interests  and  groups  such  as  the 
Audubon  Society,  the  Sierra  Club,  and,  at  the 
extreme,  the  Friends  of  the  Earth.  The  Forest 
Service  attempts  to  manage  the  land  in  a  way 


to  best  meet  the  needs  of  the  nation  and  be 
sensitive  to  both  these  concerns.  The  motto  or 
mission  of  the  Forest  Service  is  expressed  in 
"Caring  for  the  Land  and  Serving  People." [2] 

Although,  depending  on  your  point  of  view, 
the  United  States  Forest  Service  may  be 
wearing  a  white  hat  or  a  black  hat,  some  of  its 
missions  have  been  non-controversial  and 
others  have  caused  more  problems.  For 
example,  land  managed  in  the  Northeastern 
part  of  the  United  States  includes  major  areas 
trashed  by  original  users  who  logged,  mined, 
or  farmed,  the  land  to  exhaustion.  The  Forest 
Service  was  assigned  responsibility  for  these 
lands  and  restored  them  to  attractive  and 
useful  areas.  Not  too  much  to  complain  about 
in  this  situation  other  than  the  question  of  "is 
this  the  proper  role  of  the  federal  government" 
that  some  might  ask.  The  lands  managed  in 
the  South  also  do  not  present  as  many 
difficulties.  Much  of  this  area  has  been 
inhabited  for  many  years  and  several  crops  of 
timber  have  been  harvested.  The  West  was 
another  story. 

Originally,  the  West  was  settled  and/or 
exploited  by  people  who  saw  incredible 
wealth  and  extensive  resources  before  them. 
The  mountainous  forested  areas  were  vast  and 
seemed  to  be  inexhaustible.  The  Forest 
Service  was  charged  with  carefully  looking  at 
this  vast  resource  for  which  they  were 
responsible  and  identifying  appropriate  uses 
for  it.  The  Agency  identified  some  areas  as 
useful  for  logging  and  reforestation;  some  for 
hunting;  some  for  camping  and  hiking;  and 
some  for  wildlife.  These  missions  tended  to 
be  isolated  and  specialists  grew  up  around 
them.  So  did  constituencies.  Hunters  wanted 
wildlife  and  roads.  Wilderness  lovers  wanted 
places  they  could  go  and  not  hear  traffic,  guns, 
motorcycles,  chainsaws,  or  see  very  many 
people.  Major  battles  took  place  in  Congress 
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over  how  these  uses  should  be  allocated  and 
supported. 

Meanwhile,  the  chainsaws  inexorably  reduced 
the  amount  of  virgin  timber  in  the  west 
including  cutting  their  way  through  huge 
stands  of  very  old  majestic  trees.  Forest 
Service  managers  looked  at  these  stands  of 
trees  as  crops  and  thought  and  talked  of  90 
and  100  year  rotation  cycles.  Preservationists 
looked  at  the  majesty  of  these  trees  and  their 
own  finite  existence  and  that  of  their  children 
and  thought  "I  do  not  want  these  trees  cut 
down." 

The  dispute  between  the  extraction  oriented 
citizens  and  the  preservation  oriented  citizens 
placed  Forest  Service  land  managers  in  an 
awkward  situation.  Forest  Service  planners 
worked  hard  to  please  everyone.  They  failed. 
The  battles  not  only  took  place  in  Congress 
but  also  in  the  forests  where  people  chained 
themselves  to  equipment  and  trees,  drove 
huge  spikes  into  logs  that  can  and  did  kill 
when  struck  by  rapidly  spinning  chains  on 
saws  or  circular  saws  in  processing  mills.  The 
battles  took  place  in  the  courts  where  every 
aspect  of  Forest  Plans  was  challenged  on  the 
basis  of  any  deficiency,  actual  or  perceived, 
which  a  legal  mind  was  willing  to  argue. 

At  present,  the  Forest  Service  is  undergoing 
basic  structural  reorganization  but  its  current 
structure  includes  a  segment  that  manages  the 
land  it  oversees;  an  arm  that  conducts  research 
not  only  on  basic  issues  concerning  plants  and 
animals  but  also  use  of  forest  resources; 
another  arm  is  concerned  about  working  with 
states,  other  governmental  entities,  and  private 
citizens;  and  the  newest  arm  is  concerned  with 
international  and  global  forestry  issues. 

Acquisition  through  the  procurement  process 
has  always  been  an  important  way  to  support 
these  efforts.  Besides  what  is  needed  to 


support  the  organization,  the  Forest  Service 
has  traditionally  spent  its  major  sums  of 
money  on  road  construction,  reforestation,  and 
more  recently  has  begun  to  work  on 
remediation  of  environmental  problems. 

All  of  which  brings  us  to  a  small  bird:  the 
Northern  Spotted  Owl.  This  paper  is  not 
concerned  about,  nor  will  it  address,  the 
merits  of  the  scientific  argument  about  the 
Northern  Spotted  Owl  other  than  to  note  that 
the  Owl  was  alleged  to  be  in  danger  of 
extinction  because  of  the  loss  of  its  habitat 
that  were  the  large  tree  old  growth  forests  that 
were  being  reduced  fairly  rapidly.  The  Forest 
Service  multiple  use  position  was  that 
adequate  land  had  been  set  aside  for 
wilderness  and  that  only  reasonable  areas 
were  set  aside  for  timber  harvesting.  The 
opponents  argued  that  prime,  pristine,  old 
growth  was  being  destroyed  at  a  rapid  rate  and 
that  the  owl  was  scheduled  for  extinction  with 
the  trees.  During  the  1980's  the  timber 
industry  had  grown  considerably  in  the  rural 
areas  of  Oregon,  Washington,  and  Northern 
California  where  this  battle  was  being  played 
out.  By  the  early  1990's  the  situation  had 
deteriorated  to  one  where  the  courts  were 
deciding  almost  on  a  case  by  case  basis  which 
timber  harvesting  projects  were  going  to  go 
forward  in  this  area.  The  changes  were 
profound.  For  example,  in  the  impacted  area, 
the  timber  harvest  in  the  1980s  was  about  four 
billion  six  hundred  million  board  feet  per 
year.  This  was  reduced  to  approximately  two 
billion  four  hundred  million  board  feet  in  the 
1990-1992  period  and  was  projected  at  only 
one  billion  two  hundred  million  for  1994.  [3] 
This  is  a  reduction  of  almost  75%  from  what 
the  industry  in  this  area  had  come  to  expect. 
This  is  in  a  rural  part  of  the  country  where 
most  of  the  work  force  either  worked  directly 
on  harvesting  or  processing  the  wood  in  mills 
or  provided  the  necessary  community  support. 
To  say  that  this  was  a  major  impact  on  the 
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economics  of  this  part  of  the  world  would  be 
a  serious  understatement. 

INTERVENTION 

On  April  2,  1993,  the  President  convened  a 
Forest  Summit  Conference  in  an  attempt  to 
find  solutions  to  the  very  difficult  problem  of 
providing  adequate  protection  to  a  threatened 
bird  and  assisting  communities  where  that 
protection  was  causing  grievous  economic 
harm.  Besides  the  President,  the  Vice- 
President  was  there  as  were  the  Secretaries  of 
Agriculture  (Mike  Espy)  and  Interior  (Bruce 
Babbitt).  Members  of  Congress  and  many 
concerned  local  citizens  were  also  there,  of 
course.  This  meeting  resulted  in  the  creation 
of  a  Memorandum  of  Understanding  signed 
by  the  President,  Congressional 
Representatives,  and  other  concerned  parties. 
The  basic  intent  of  this  Memorandum  of 
Understanding  was  to  increase  economic 
activity  in  the  affected  communities  to  make 
up  for  the  economic  loss  caused  by  the  decline 
in  the  timber  harvest. 

After  the  Conference,  two  actions  began.  One 
of  these  was  national  and  the  other  was  local. 
Congress  provided  funds  to  accomplish  the 
goals  of  the  agreement  and  a  structure  of 
community  based  organizations  were 
established  and  began  working  to  decide  what 
needed  to  be  done.  The  local  teams  went 
under  the  general  title  of  "Community 
Economic  Revitalization  Teams"  (CERT)  and 
there  was  an  organizational  system  of  CERTs 
created.  Their  function  was  to  determine  local 
needs  and  coordinate  with  Federal  Agencies  to 
see  that  those  needs  were  met.  These  groups 
were  created  in  November  of  1 993  and  were 
organized  by  state  and  region.  There  was  a 
multi-agency  command  for  this  structure 
established  in  Washington,  D.C.  The 
Governors  of  California,  Oregon,  and 
Washington  as  well  as  the  Chair  of  the  Multi- 
Agency  command  signed  the  final  plan  for  the 


Community  Economic  Revitalization  Teams 
in  December  of  1993.  There  was  a  major 
issue  in  working  with  these  groups  about  the 
Federal  Advisory  and  Committee  Act.  It  was 
resolved  and  is  not  the  topic  of  this  paper 
other  than  to  note  that  trying  to  involve  local 
communities  in  Federal  efforts  often  has 
unanticipated  consequences. 

The  national  issue  was  providing  funding  and 
a  mechanism  for  spending  the  money.  This 
was  a  major  initiative  involving  many  federal 
agencies.  Contract  expenditures  were  but  a 
small  portion  of  the  $300,000,000  available 
for  this  project.  Congress  provided  well  over 
half  of  these  funds  to  the  Department  of 
Agriculture's  Rural  Development 
Administration.  The  funds  for  the  Jobs  in  the 
Woods  program  for  the  Forest  Service  was 
around  $20,000,000  or  just  less  than  7%  of  the 
total  expenditure.  In  addition,  the  Bureau  of 
Land  Management  was  to  spend  $5,000,000 
and  the  Fish  and  Wildlife  Service  and  the 
Bureau  of  Indian  Affairs  $1,000,000  each. 
This  total  of  $27,000,000  then  was  just  under 
1 0  percent  of  the  total  program.  Besides  the 
agencies  already  cited,  funds  for  the  program 
were  provided  through  Housing  and  Urban 
Development,  the  Small  Business 
Administration  (loan  guarantees  only),  the 
Department  of  Labor  (Title  III  money  for  the 
Job  Training  Partnership  Act),  the  Department 
of  Commerce,  and  the  Environmental 
Protection  Agency.  [4] 

Acquisition  Issues:  In  late  1993,  the 
procurement  mission  began.  The  initial  Forest 
Service  mission  was  to  support  local 
communities  by  contracting  for  watershed 
restoration  services  in  the  amount  of 
$20,000,000  to  help  communities  in  the 
coastal  mountain  ranges  from  the  Canadian 
border  into  Northern  California  by  the  end  of 
fiscal  year  1994.  The  individual  communities 
were  expecting  that  the  money  would  be  spent 
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locally.  They  were  expecting  that  the 
contractors  would  be  local  and  that  the 
workers  will  be  local.  They  expected  that  the 
money  would  be  flowing  very  quickly  since 
they  needed  it  immediately.  Clearly  smaller 
projects  would  not  be  a  problem. 
Acquisitions  of  less  than  $25,000  could  be 
obtained  with  limited  local  competition  and  a 
reasonable  expectation  that  the  workers  would 
be  of  local  origin.  Larger  acquisitions 
presented  more  difficulty. 

There  were  really  two  major  issues  here:  who 
the  contractors  might  be  and  where  would  the 
workers  come  from.  The  two  extremes  of  the 
argument  were  that  the  workers  should  not 
only  all  be  local  but  also  displaced  timber 
workers  and  that  contractors  should  all  be 
local  to  insure  that  the  money  would  stay  in 
the  community.  The  other  extreme  was  that 
the  Competition  in  Contracting  Act  required 
national  advertisement  of  formal  procurement 
with  no  restriction  on  the  workers  other  than 
that  they  be  legal. 

The  issue  of  the  workers  was  fairly  easily 
resolved.  There  was  some  strong  sentiment 
for  guaranteeing  that  the  work  force  would  be 
displaced  timber  workers.  However,  that 
argument  failed  because  any  mechanism  to 
attempt  to  insure  this  would  either  collapse  of 
its  own  weight  or  be  excessively  burdensome. 
The  skill  mix  was  not  always  good.  Sawyers 
and  mill  workers  were  not  always  willing  to 
become  laborers  on  construction  and  other 
watershed  restoration  projects.  Therefore,  the 
concept  of  trying  to  develop  a  system  to  target 
who  could  be  hired  on  these  projects  was 
abandoned. 

The  issue  concerning  the  geographic 
qualification  of  the  contractors  needed  more 
careful  attention.  The  Forest  Service  was  used 
to  limiting  competition  under  the  Competition 
in  Contracting  Act  when  the  situation  was 


urgent  and  time  did  not  permit  full  and  open 
competition.  There  were  advocates  for  using 
this  authority  in  this  situation.  However,  the 
argument  prevailed  that  although  this  was  a 
difficult  situation  for  many  U.S.  Citizens  in 
one  part  of  the  country,  there  was  no  serious 
threat  to  the  nation  or  to  the  lives  of  its 
citizens.  Arguments  that  there  were  only  one 
or  a  few  sources  that  could  do  the  work  were 
patently  untrue  so  that  did  not  merit  a  great 
deal  of  consideration.  No  serious  thought  was 
given  to  arguing  that  there  was  a  need  to 
maintain  any  particular  industrial  base  as  a 
way  to  limit  competition.  This  left  exception 
7  to  the  Competition  in  Contracting  Act  —  the 
Public  Interest  Waiver.  It  made  a  great  deal  of 
sense  and  seemed  to  fit  the  situation  very  well 
but  using  it  had  several  serious  drawbacks. 
First,  it  required  Secretarial  approval  and  was 
something  that  had  rarely  been  used.  Its  few 
occasions  of  use  had  been  accompanied  by 
problems.  Second,  it  required  Congressional 
notification  with  thirty  days  of  waiting  before 
beginning  work  after  that  took  place.  Third,  it 
required  this  approval  to  be  done  on  a  specific 
project  basis.  Although  the  authority  fit  the 
situation  very  well,  it  was  easy  to  guess  that 
this  situation  was  not  quite  what  the  drafters 
of  the  law  had  in  mind.  Most  likely  it  was 
intended  for  a  situation  where  it  was  in  the 
national  interest  to  award  a  contract  to  a 
specific  vendor  when  the  need  for  an 
industrial  base  could  not  be  justified  but  it 
would  be  in  some  unspecified  way  in  the 
national  interest  to  take  that  act. 

Exception  7:  After  a  great  deal  of  internal 
debate  and  discussions  between  procurement 
personnel  at  field  locations  and  the 
Washington  Office  of  the  Forest  Service, 
Department  of  Agriculture  General  Counsel 
personnel,  and  the  project  managers,  it  was 
decided  to  seek  Secretarial  approval  for  an 
exception  to  CICA  based  on  national  interest. 
This  was  not  taken  lightly  but  there  was  good 
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expectation  of  approval  since  the  Secretary 
had  been  party  to  the  original  Memorandum 
of  Understanding  so  was  aware  of  and 
interested  in  the  project.  Also,  Secretary  Mike 
Espy  had  taken  an  uncommon  interest  in  the 
subject  of  procurement  as  shown  by  his 
calling  most  of  his  assistant  Secretary's 
together  in  a  meeting  room  with  most  of 
Department's  procurement  personnel  in  the 
Washington,  D.C.  area  and  declaring  his 
interest  in  procurement  and  then  convening  a 
task  force  to  improve  Departmental 
procurement. 

However,  the  issue  was  not  clear  cut.  What 
action  to  take  and  exactly  how  to  take  it 
involved  many  discussions  and  memos 
between  local  managers,  national  project 
managers,  Departmental  attorneys,  and  the 
procurement  management  community. 
Finally  in  April  of  1994,  the  Chief  of  the 
Forest  Service  brought  the  recommendation  to 
a  head  when,  in  a  letter  to  the  Secretary,  he 
stated: 

I  have  determined  that  it 
would  be  more  appropriate  to 
recommend  to  the  Secretary  of 
Agriculture  to  seek  a  public 
interest  waiver  for  these 
projects  pursuant  to  41  U.S.C. 
(paragraph)  253(c)(7).  A 
public  interest  waiver  offers 
the  most  flexible  means  of 
accomplishing  the  direction 
laid  down  by  President  Clinton 
and  Congress  to  direct  award 
of  these  watershed  restoration 
projects  to  communities 
affected  by  decreased  timber 
harvesting.  [5] 

Clearly  we  could  not  have  the  Secretary 
individually  approve  each  of  the  hundreds  of 
projects  that  the  Agency  intended  to 


accomplish.  Therefore,  the  Agency  decided  to 
obtain  the  listing  of  all  the  projects  that  were 
to  be  accomplished.  Although  the 
Revitalization  Teams  had  been  working  with 
the  local  procuring  units,  the  specific  projects 
were  far  from  being  ready  for  clear 
identification.  Since  the  work  field  season 
was  rapidly  evaporating  and  the  Secretarial 
approval  and  Congressional  notice  had  yet  to 
be  accomplished  it  was  vital  that  the  projects 
be  identified  and  provided  to  the  Washington 
Office  of  the  Forest  Service  for  consolidation 
and  submittal  to  the  Department  of  the 
Agriculture.  Although  this  was  a  difficult 
task,  it  is  not  particularly  instructive  here 
except  to  note  that  in  situations  like  this  it  is 
useful  to  stay  flexible  and  keep  a  sense  of 
humor. 

On  April  11,  1994,  the  Secretary  of 

Agriculture  approved  the  request  from  the 
Forest  Service  and  the  necessary  notification 
was  hand  carried  to  the  Speaker  of  the  House 
of  Representatives  and  the  President  of  the 
Senate. 

Freefall:  Oops.  It  quickly  became  apparent 
that  taking  exception  to  the  Competition  in 
Contracting  Act  meant  that  major 
requirements  of  the  Federal  Acquisition 
Regulations  had  just  disappeared  with  no 
good  precedent  on  how  to  proceed.  This  was 
both  liberating  and  somewhat  daunting.  The 
concerns  came  from  having  to  insure  that 
business  would  be  accomplished  properly  in 
an  uncertain  environment.  The  liberating 
feeling  came  from  knowing  that  it  was  now 
possible  to  take  the  best  possible  actions  to 
meet  the  intent  of  those  that  had  provided  the 
funds  while  still  maintaining  the  integrity  of 
the  procurement  process.  The  Forest  Service 
has  long  maintained  that  the  keys  to  the 
federal  acquisition  system  are  fairness  and 
flexibility.  The  Agency  now  had  the 
opportunity  to  prove  that  it  could  operate  in 


510 


this  fashion. 

As  discussed  above,  the  Agency  had  dropped 
the  idea  of  trying  to  limit  who  was  eligible  to 
be  hired  by  contractors  but  still  needed  to 
decide  who  could  compete  for  these  projects. 
There  was  no  sentiment  for  including 
contractors  that  were  not  near  the  impacted 
area.  There  would  have  been  little  reason  for 
the  exception  if  competition  had  been  open  to 
all.  The  only  reason  would  have  been  time 
and  although  that  was  significant  it  was  only 
one  small  part  of  the  need.  The  three  major 
alternatives  were  to  contract  with:  1)  firms 
located  in  the  general  area  of  impact;  2)  firms 
located  only  in  the  impacted  counties;  and  3) 
firms  locate  in  the  county  where  each  specific 
project  was  to  be  accomplished. 

1.  Although  the  areas  of 
greatest  impact  were  clearly 
identified,  the  entire  area  was 
hurt  by  the  reduction  in  timber 
harvesting  and  processing. 

Also,  although  workers  do 
travel  great  distances  to 
accomplish  Forest  Service 
contracts,  most  of  the 
contractors  work  in  a  general 
localized  area  and  it  might  be 
difficult  to  explain  to 
contractors  in  Eastern  Oregon 
why  they  could  not  compete 
for  contracts  where  they 
normally  did.  The  exclusion 
could  be  more  easily  justified 
to  a  contractor  who  lived  in 
some  Southeastern  state,  for 
example. 

2.  The  Agency  did  have  the 
advantage  of  having  a  clearly 
defined  area  of  affected 
counties.  Making  the 
assumption  that  all  contractors 


residing  in  any  of  these 
counties  in  the  three  state  area 
could  compete  for  the  work 
permitted  maximum 
competition  in  the  impacted 
area.  This  general  impact  is 
important  since  one  facet  of 
this  evaluation  was  that  there 
was  not  necessarily  a 
connection  between  where  the 
projects  were  located  and 
where  the  nearest  communities 
were.  For  example,  if  the 
work  were  to  be  accomplished 
in  a  forested  area  close  to  a 
community,  the  community 
clearly  would  have  a  major 
interest  in  having  the  work 
accomplished  by  local 
displaced  contractors  and 
workers.  On  the  other  hand, 
there  were  situations  where  the 
work  was  to  be  accomplished 
in  forested  areas  where  there 
were  no  or  few  contractors  or 
residents.  In  this  situation, 
more  latitude  needed  to  be 
taken. 

3.  The  most  restrictive 
approach  would  be  to  limit  the 
contractor  base  to  the  local 
community  or  county  and  not 
allow  outside  contractors  to 
compete.  This  was  in  some 
ways  most  attractive  to  some 
local  communities  and 
Revitalization  Teams.  If  this 
were  done,  dollars  spent  in  the 
local  community  would  stay  in 
the  local  community.  There 
was  strong  feeling  that  this 
was  what  was  intended  and 
what  should  be  done.  It  had 
the  disadvantages  identified 
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above  of  not  always  having  a 
nice  fit  with  where  the  work 
was  to  be  accomplished  and 
where  the  displaced  workers 
might  live.  It  also  had  the 
major  disadvantage  of  needing 
to  draft  very  specific  contract 
language  for  each  project  and 
each  community. 

One  Forest  Service  issue  was  to  decide  which 
of  these  three  approaches  to  choose  or  to 
decide  if  there  were  other  more  attractive 
alternatives.  It  was  clearly  the  intent  of  the 
legislation  to  direct  funds  to  the  local  affected 
communities.  Clearly  the  intent  of  Federal 
Procurement  is  to  maximize  competition  in 
every  situation.  The  Forest  Service  chose  the 
second  option  described  above  of  restricting 
competition  to  firms  located  within  the 
affected  counties  but  allowing  any  firms 
within  the  affected  area  to  compete.  [6] 

Operations:  Because  the  project  lists  were 
provided  to  Congress,  every  project  was 
identified  and  known  to  interested  Members 
of  Congress  and  anyone  else  who  was 
interested.  Contracting  Officers  posted  this 
list  of  projects  in  contracting  offices 
throughout  the  three  state  area.  However, 
contracting  officers  were  free  to,  and  did, 
concentrate  on  obtaining  competition  only 
from  local  contractors.  If  other  firms  in 
impacted  portions  of  the  three  state  area 
requested  the  opportunity  to  compete  on  any 
specific  package,  their  proposals  were 
accepted  and  evaluated.  In  a  sense,  these 
procurements  were  handled  much  like  small 
purchases.  Contracting  Officers  solicited 
Competition  within  local  trade  areas  but 
accepted  proposals  from  any  interested 
proposer. 


CONCLUSIONS 

Although  the  procurement  system  takes  many 
lumps  it  does  work  very  well  and  has  been 
designed  to  provide  a  great  deal  of  flexibility 
to  those  who  are  willing  to  work  with  it. 
There  seems  to  be  at  least  two  different 
procurement  systems.  The  one  is  the  legal 
requirement  involving  entering  into, 
administering,  and  closing  contracts.  That 
system  can,  and  usually  does,  work  very  well. 
The  frequently  maligned  system  is  one  where 
everyone  from  Congress  and  the  President 
down  to  the  local  manager  try  to  use  it  for 
their  own  purposes.  For  example,  almost 
everyone  who  works  with  the  system  is  aware 
of  national  impositions  such  as  setting 
minimum  wages  for  those  employed  on 
contracts  to  trying  to  increase  the  soundness 
of  the  system  through  the  strictures  of  the 
Procurement  Integrity  Act.  Systems  are  in 
place  to  insure  that  a  fair  proportion  of 
business  is  provided  to  the  small  business 
community  and  even  more  specifically  to 
disadvantaged  firms  and  others  specifically 
targeted.  At  the  local  level  in  small 
communities,  a  dwindling  number,  hopefully, 
of  benighted  managers  prefer  to  have 
maximum  flexibility  in  the  placement  of  small 
purchases  so  they  can  guarantee  that  troubled 
local  businesses  survive  and  that  no  one 
unduly  prospers  from  the  federal  procurement 
process.  At  its  best,  this  may  be 
understandable,  if  patriarchal.  At  its  worst,  it 
creates  a  power  position  for  the  local  federal 
government  manager  that  could  lend  itself  to 
abusive  control.  ("It  would  be  a  shame  Henry 
if  you  took  that  position.  It  would  certainly 
damage  us.  Oh,  by  the  way,  how  much 
business  did  we  do  in  your  store  last  year. 
Y ou  know,  one  nice  thing  about  the  small 
purchase  system  is  that  we  have  a  great  deal  of 
flexibility  concerning  the  vendors  that  we 
patronize.")  The  point  is  a  simple  one.  The 
procurement  system  as  practiced  by 
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procurement  professionals  can  work  very 
well.  In  fairness,  we  can  sometimes  gum  it 
up.  However,  THE  PROCUREMENT 
SYSTEM  as  defined  by  everything  that 
touches  procurement  (management,  personal, 
and  economic  interests)  from  requirement 
initiation  to  final  payment  can  have  many 
problems  from  dealing  with  people  who 
develop  over  controlling  specifications, 
working  through  massive  controls  to  insure 
that  business  is  equitably  distributed,  and 
finally  to  fending  off  those  that  want  to  have 
the  work  accomplished  by  some  specific 
individual  or  firm.  In  the  Jobs  in  the  Woods 
example,  there  was  clear  Congressional  intent 
to  support  local  communities  affected  by 
severe  cutbacks  in  the  logging  industry.  As  is 
so  often  the  case,  that  intent  was  not  complete 
with  the  necessary  changes  in  procurement 
legislation  to  make  that  work  smoothly. 
However,  the  procurement  system  still  has  all 
the  necessary  tools  so  that  even  in  these 
circumstances  the  system  can  work  well. 

Congress  clearly  intended  and  the 
Administration  wanted  to  help  through  the 
procurement  process  a  large  number  of 
communities  that  had  been  negatively  affected 
by  the  reduction  of  timber  harvesting  because 
of  environmental  concerns.  Although  the 
Competition  in  Contracting  Act  requires 
maximizing  competition  it  provides  many 
situations  where  some  liberties  can  be  taken 
with  the  overall  principle.  This  acquisition 
demonstrates  that  if  an  Agency  is  committed 
to  a  program  it  deems  to  be  in  the  public 
interest,  the  procurement  process  does  not 
have  to  be  a  barrier  to  accomplishing  those 
objectives. 

A  standard  accusation  against  the  procurement 
system  is  that  it  takes  too  long.  Sadly,  that  is 
often  true.  However,  as  discussed  above,  that 
is  more  the  fault  of  the  PROCUREMENT 
SYSTEM  than  it  is  of  the  procurement 


system.  In  this  example,  in  less  than  a  year 
large  numbers  of  projects  were  identified; 
discussions  were  held  with  large  numbers  of 
local  communities  concerning  what  work 
would  be  of  the  most  interest  and  value  to 
them;  a  Secretarial  waiver  was  obtained  by 
two  major  civilian  agencies;  Congress  was 
advised  as  required;  projects  were  advertised 
and  solicitations  issued;  contracts  were 
awarded  and  work  was  begun;  and,  in  some 
cases,  completed.  There  is  no  question  but 
what  those  involved  in  this  process  would 
have  liked  more  time  for  reflection  and  review 
of  this  process.  However,  that  time  was  not 
available  the  tasks  were  all  accomplished 
within  the  allotted  time.  (Could  Mr. 
Parkinson  have  been  on  to  something?) 

Interestingly,  in  light  of  our  discussion  of  the 
problems  of  the  larger  or  the  smaller  portions 
of  the  procurement  system,  one  of  the  biggest 
difficulties  was  defining  what  tasks  were  to  be 
done.  Forest  Service  managers,  like  many 
others,  are  excellent  at  planning  what  will  be 
done  over  the  next  80-year  timber  cycle  but 
not  so  good  at  planning  what  procurement  will 
be  done  during  the  next  fiscal  year(or  even  in 
the  current  fiscal  year).  One  thing  that  was 
clear  early  in  this  process  was  that  the  Agency 
could  not  expect  to  go  repeatedly  to  the 
Secretary  of  Agriculture  for  his  approval. 
Exception  7  to  the  Competition  in  Contracting 
Act  does  require  case  by  case  approval. 
Therefore,  the  Forest  Service  needed  to 
identify  all  the  projects  that  it  intended  to 
accomplish  through  the  Jobs  in  the  Woods 
program  during  Fiscal  Year  1994.  Although 
this  could  be  perceived  as  a  problem  in  the 
procurement  system  it  would  be  as  easy  to 
think  of  it  as  a  problem  in  knowing  what  the 
Agency  really  intended  to  accomplish.  This 
process  is  again  being  conducted  for  Fiscal 
Year  1995  as  this  paper  is  being  written. 
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The  Jobs  in  the  Woods  program  did  a  good 
job  of  expending  the  funds  set  aside. 
Although  the  Forest  Service  spent  the  most 
money,  on  a  percentage  basis  its  results  were 
not  as  good  as  the  other  Agencies.  The 
Bureau  of  Land  Management  obligated  the 
$5,000,000  that  was  available  to  it  and  Bureau 
of  Indian  Affairs  and  the  Fish  and  Wildlife 
Service  each  obligated  the  $1,000,000 
available  to  them.  However,  the  Forest 
Service  was  only  able  to  obligate  $1 1,284,000 
of  the  $20,000,000  available  to  it.  [7]  This 
was  due  in  part  to  the  larger  size  of  the 
program  but  more  specifically  to  the 
horrendous  fire  season  experienced  during  the 
summer  and  fall  of  1994.  Most  Forest  Service 
contracting  resources  were  totally  absorbed  in 
providing  fire  support  and  the  Jobs  in  the 
Woods  program  as  well  as  everything  else  had 
to  wait. 

There  was  one  very  negative  aspect  to  this 
otherwise  successful  program.  Procurement 
targeted  to  specific  communities  provides  a 
new  opportunity  for  "pork  barrel"  politics. 
Traditionally,  "pork  barrel"  politics  has 
involved  grants  to  specific  institutions  or 
firms  or  even  money  for  a  specific  project  in 
a  geographic  location.  However,  in  most  of 
these  instances  there  has  been  no  attempt  to 
insure  that  the  citizens  of  the  area  were  the 
only  ones  that  could  bid  or  work  on  the 
project.  [8]  I  think  there  is  a  general 
understanding  that  large  companies  have 
sometimes  spread  subcontracts  widely  across 
the  country  insuring  that  major  portions  would 
be  in  key  Congressional  districts.  This 
normally  tended  to  be  covert  as  opposed  to 
overt.  The  firms,  at  least  in  public,  did  not  say 
their  weapons  system  should  be  selected,  not 
because  it  was  the  best,  but  because  the  major 
subcontracts  were  in  key  Congressional 
districts.  Jobs  in  the  Woods  effectively 
defined  a  geographic  area  of  concern  and  said 
only  contractors  located  within  this  area  can 


compete.  This  could  be  a  major  precedent  in 
the  base  closure  arena  for  the  Department  of 
Defense.  To  date  the  Secretary  of  Defense  has 
not  chosen  to  set  aside  procurement  for  Los 
Angeles  or  any  other  area  because  of  the 
negative  impact  of  base  closures  or  reductions 
in  expenditures  in  a  specific  geographic  area. 
The  Jobs  in  the  Woods  program  set  aside  a 
rural  portion  of  the  country  and  closed  certain 
contracts  in  that  geographic  area  to  most  of 
the  contractors  in  this  country.  This  could 
have  serious  implications  for  rural  legislators. 
If  procurement  dollars  start  being  allocated  by 
geographic  area  this  will  greatly  complicate 
the  process  and  the  American  people  will  be 
the  losers  in  two  major  ways:  Competition 
will  be  greatly  reduced  and  opportunity  for 
mobile  contractors  will  similarly  be  reduced. 
This  is  not  a  concern  for  most  of  the  people 
involved.  For  example,  the  Vice-President 
promoted  this  approach  as  a  good  way  to  do 
business:  "This  is  the  kind  of  interagency, 
intergovernmental  cooperation  I  like  to  see  .  . 

. .  We  are  part  of  the  successful  development 
of  rural  America  here  that  other  distressed 
communities  can  look  to  as  an  example  of 
how  to  get  things  done."  [9] 
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3.  "The  Forest  Plan,  In  Brief,"  Summary 
prepared  for  Interagency  meeting,  October  20 
and  21,  1993,  Portland  Hilton,  Portland, 
Oregon. 
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4.  "Economic  Adjustment  Initiative  . . .  under 
President  Clinton's  Forest  Plan,"  Annual 
Report.  October  20,  1994,  pp.  5-9. 

5.  Thomas,  Jack  Ward,  Chief  of  Forest 
Service  letter  to  Regional  Foresters  of  the 
Pacific  Northwest  and  Pacific  Southwest 
Regions,  Reply  to  2520,  Public  Interest 
Waiver  for  Watershed  Restoration  Projects, 
April  4, 1994. 

6.  Reimers,  Mark  A.  (for  Chief  of  Forest 
Service)  letter  to  Regional  Foresters  of  the 
Pacific  Northwest  and  Pacific  Southwest 
Regions,  Reply  to  6300,  Authority  to  Limit 
Competition,  May  18,  1994. 

7.  "Economic  Adjustment  Initiative  .  . .  under 
President  Clinton's  Forest  Plan,"  Annual 
Report.  October  20,  1994,  p.  A2. 

8.  This  discussion  on  limiting  work  to  local 
areas  is  of  current  interest  in  light  of  renewed 
interest  in  reviewing  the  Davis  Bacon  Act. 
It  has  been  argued  that  the  Act's  original 
premise  was  to  discourage  the  importation 
of  cheaper  labor  from  other  parts  of  the 
country  to  the  industrialized  cities  of  the 
North.  This  then  reflects  a  long  time  interest 
in  keeping  dollars  in  the  pockets  of  local 
labor. 

9.  "Economic  Adjustment  Initiative  . . .  under 
President  Clinton's 

Forest  Plan,"  Annual  Report.  October  20, 
1994,  p.  5. 
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Cooperative  Acquisition  Projects 
in  the  Pacific  Rim 


BACKGROUND 

In  1992  the  Defense  Systems 
Management  College  (DSMC)  began  a 
study  of  international  cooperative  defense 
acquisition  projects  with  the  United  States 
and  countries  in  the  Pacific  Rim.  This  was 
the  third  of  three  related  research  studies  of 
cooperative  acquisition  projects  conducted 
during  the  past  ten  years.  The  first  and 
second  were  studies  of  US/NATO-Europe 
projects  completed  in  19851  and  199Q2. 
Responding  to  increasing  student  demand 
for  information  on  PACRIM  projects, 
DSMC  took  the  lead  for  the  third  research 
study. 

The  research  objectives  were  as  follows: 

.  Describe  the  current  reality  of 
cooperative  projects  in  the 
Pacific  Rim. 

Determine  the  Rx,  or 
prescription,  for  success  by 
identifying  barriers  to  and 
facilitators  of  cooperation. 


1  C.  Michael  Farr,  "An 
Investigation  of  Issues  Related 
to  Success  or  Failure  in  the 
Management  of  International 
Cooperative  Projects,"  Ph.D. 
diss.,  University  of  North 
Carolina,  Chapel  Hill,  1985. 

2  C.  Michael  Farr, 
"Managing  International 
Cooperative  Projects:  "Rx  for 
Success,"  Chapter  6,  Global 
Arms  Production.  University 
Press  of  America,  Inc.,  Lanham, 
MD,  1992. 


Examine  similarities  and 
differences  between  PACRIM 
and  NATO-Europe  Projects. 

The  PACRIM  study  progressed  in 
four  phases  employing  the  tailored 
questionnaires  and  methodology  developed 
during  the  two  previous  studies.  The  four 
phases  are  shown  in  Table  1. 


Study  Phases 

Participating 

Organizations 

I. 

Establish  terms  of 
reference  and 
notes  on 
cooperation 

Office  of  Secretary 
of  Defense, 

Defense  Security 
Assistance 

Agency,  and 

Service  Staffs 

n. 

Notes  on 
cooperation 

Allies  -  Embassies 

m. 

International 
Acquisition  Topics 

U.S.  Project 

Offices 

IV. 

Internationa! 
Acquistion  Topics 

Allies  Project 
Offices  and  U.S. 
In-Country 
Personnel 

Table  1.  Study  Phases  and  Participating 
Organizations 

Phase  I  was  conducted  to  identify  the 
PACRIM  countries  and  projects  of  interest. 
As  a  result,  the  scope  of  the  study  was 
narrowed  to  include  only  the  nations  of 
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Australia,  Japan,  and  South  Korea.3  While 
others  were  mentioned,  it  is  with  only  these 
three  that  the  Department  of  Defense  has 
the  legal  authority  to  enter  into  a 
cooperative  acquisition.  A  cooperative 
acquisition  must  be  jointly  managed  and 
equitably  (or  equally)  funded  by  the 
participating  nations,  and  there  must  be  an 
international  agreement  setting  forth  the 
terms  and  conditions  of  the  project.  This  is 
normally  called  a  Memorandum  of 
Understanding  or  Agreement  (MOU  or 
MOA).  All  international  projects  with  any 
other  PACRIM  nation,  cooperative  or 
otherwise,  must  use  Foreign  Military  Sales 
(FMS)  procedures.  The  projects  identified 
for  study  are  shown  in  Table  2.  A  good 
balance  was  attained  with  at  least  two 
projects  from  each  Military  Department 
(plus  one  Defense  Agency  project),  and  at 
least  two  from  each  nation  of  interest. 


3  The  nations  are  listed 
in  alphabetical  order;  no 
precedence  is  implied.  While 
Canada  could  be  considered  as  a 
PACRIM  country,  it  is  normally 
included  with  the  NATO-Europe 
countries. 


The  Projects  Studied 


Project 

Military 
Department 
or  Defense 
Agency 

Allied  Nation 

Next 

Generation 

Support 

Fighter  (aka 
FS-X) 

Air  Force 

Japan 

Ducted  Rocket 
Engine 

Army 

Japan 

Coastal 

Harbor 

Defense 

Navy 

South  Korea 

Ammunition 

Storage 

Technology 

Army 

South  Korea 

Digital  Chart 
of  the 

World 

Defense 

Mapping 

Agency 

Australia 

Radar 

Activities 

Air 

Force/Navy 

Australia 

MK-53  Off- 
Board 

Active  Decoy 
(aka  Nulka) 

Navy 

Australia 

Table  2. 


Phases  I  and  phase  II  of  the  study 
were  combined  to  produce  general  and 
country-specific  notes  on  cooperation. 
These  are  covered  in  the  next  section. 

Phase  HI  conducted  during  1993  and 
Phase- -IV-  conducted  during  1994  focused 
specifically  on  the  PACRIM  projects. 
Interviews  were  conducted  with  both  U.S. 
and  allied  in-country  representatives  of 
each  of  the  project  offices.  Logistical 
support  for  the  trip  and  additional 
information  on  the  projects  was  provided 
by  U.S.  in-country  personnel  from  the  Joint 
U.S.  Military  Advisory  Group  (JUSMAG) 
in  Seoul,  South  Korea,  the  Mutual  Defense 
Assistance  Office  (MDAO)  in  Toyko, 
Japan,  and  the  Office  of  Defense 
Cooperation  (ODC)  in  Canberra,  Australia. 
The  allied  project  offices  visited  are  shown 


at  Table  3.  These  included  the  Agency  for 
Defense  Development  (ADD)  laboratories  in 
Chinhae  and  Taejon,  South  Korea;  the 
Technical  Research  and  Development 
Institute  (TRDI)  in  Toyko,  Japan;  and  the 
Jindalee  Operational  Radar  Network 
(JORN)  Program  Management  Office,  the 
Nulka  Project  Office,  and  the  Director  of 
Survey  -  Army  in  Canberra,  Australia. 
Since  the  detailed  questionnaires  provided 
to  both  U.S.  and  allied  project  offices  were 
identical,  and  the  questionnaires  were 
similar  to  those  of  the  U.S./NATO-Europe 
project  studies,  comparisons  between  U.S. 
and  allied  views  on  cooperative  acquisition 
can  be  made,  as  well  as  some  general 
comparisons  between  PACRIM  and  NATO 
projects. 


Project  Organizations  Visited 


Project 

U.S.  Organization 

Allied 

Org 

Coastal 

NHL, 

ADD, 

Harbor 

Washington,  D.C. 

Chinhae 

Def. 

TCES,  Savanna, 

ADD, 

Ammo 

Storage 

IL 

Taejon 

Tech 

Support: 

JUSMAG-K 

FS-X 

F-16  SPO, 

TRDI, 

Ducted 

WPAFB,  OH 

Tokyo 

Rocket 

MICOM, 

TRDI, 

Engine 

Huntsville,  AL 

Support:  MDAO 

Tokyo 

Radar 

ESC,  Hanscom 

JORN 

Activities 

AFB,  MA 

PMO, 

Nulka 

TAD  PEO, 

Canberra 

Crystal  City,  VA 

Nulka 

Digital 

DMA,  Fairfax, 

PO, 

Chart  of 
the  World 

VA 

Canberra 

Support:  ODC 

Dir.  of 
Survey  - 
Army, 

Canberra 

Table  3. 


One  of  the  great  difficulties  in  a  study 
of  the  Rx  for  project  success  is  in 


determining  the  definition  of  success.  Early 
on  in  the  study  a  simple  definition  was 
developed.  Success  was  defined  as  having 
a  signed  MOU,  national  funding  provided 
from  the  participating  nations,  the  project 
initiated,  and  no  withdrawal  or  termination 
due  to  unresolved  problems.  All  the 
projects  studied  met  this  definition  at  both 
the  beginning  and  end  of  the  study.  The 
simple  definition  and  the  nature  of  the 
projects  available  for  study,  for  the  most 
part,  led  to  a  focus  on  the  very  preliminary 
stage  of  the  acquisition  process. 

NOTES  ON  COOPERATION 
General 

The  first  and  foremost  general  note 
that  must  be  kept  in  mind  is  that  there  is  no 
equivalent  to  NATO  in  the  Pacific  Rim. 
This  means  that  there  is  none  of  the  vast 
NATO-type  infrastructure  in  place  to 
support  cooperative  activities  with  Pacific 
Rim  nations.  Therefore,  with  few 
exceptions,4  our  cooperative  acquisition 
projects  with  Australia,  Japan  and  South 
Korea  are  conducted  bilaterally,  and  will 
remain  so  for  the  foreseeable  future.  The 
U.S.  enjoys  favorable  defense  trade 
balances  with  the  three  nations,  and  is 
therefore  pressured  to  give  generous  terms 
in  cooperation.  Furthermore,  one  should 
not  be  fooled  into  stereotypical  thinking. 
Each  nation  is  different  —  Japan  is  not  like 
Korea;  Australia  is  different  in  many  ways 
from  the  U.S.  There  can  be  enormous 
cultural  differences  between  each  nation  as 
well  as  management  styles  and  motivations 
for  cooperative  acquisition.  One  also 
should  beware  of  "European  Strings," 
which  may  tie  our  hands  in  the  Pacific  Rim 
because  of  prior  commitments  made  in 
European  projects.  Interestingly,  there  was 


The  exception  is 
Australia,  which  participates 
both  bilaterally  and 
multilaterally  with  the  U.S., 
United  Kingdom,  Canada,  and  New 
Zealand. 
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a  perception  among  the  U.S.  staff  personnel 
interviewed  that  our  system  was  the  most 
problematic  when  it  comes  to  cooperative 
acquisition.  This  was  especially 
pronounced  in  our  legal  system  (eg. 
treatment  of  intellectual  property  rights) 
and  acquisition  system  (eg.  competition 
policies). 

Australia 

Australia  is  geographically  a  Pacific 
Rim  nation,  but  is  heavily  populated  with 
transplanted  Europeans.  For  the  U.S., 
Australia  is  culturally  the  easiest  nation  to 
work  with  in  the  Pacific  Rim,  if  not  in  the 
world.  Further  smoothing  relations, 
Australia  is  not  viewed  as  a  competitor  to 
the  United  States,  whether  economically  or 
in  the  defense  export  market.  The 
Australian  defense  budget  is  very  small  in 
comparison  with  that  of  the  U.S.,  but  they 
maintain  a  relatively  large  portion  for 
research.  Therefore,  Australia  can  be 
viewed  as  strong  on  research,  but  weaker 
on  development.  They  seek  more 
cooperative  projects  with  the  U.S.  to 
develop  outlets  for  their  research 
technology,  and  to  attain  rational 
production  quantities.  Their  rationale  for 
cooperation  is  to  access  technology, 
promote  their  technology,  realize  economics 
of  scale,  promote  interoperability,  and 
encourage  industrial  participation  with 
"residual"  capability.  All,  except  the  last, 
are  identical  to  U.S.  motivations  for 
cooperation.  Residual  capability  refers  to 
an  Australian  motivation  to  further  build 
their  industrial  base,  and  to  examine  every 
potential  cooperative  project  for  the 
industrial  capability  retained  in  Australia 
after  project  completion. 

Australia  explores  cooperative  project 
opportunities  through  a  variety  of  ways. 
These  include  the  structural  process 
(attaches,  exchange  officers,  etc.), 
multilateral  forums  (ABCA,  TTCP,  5 
Nations,  etc.),  senior  national 
representative  meetings,  and  project  teams 


specially  formed  to  examine  the  pros  and 
cons  of  the  cooperative  project. 

Australia  cooperates  with  many 
nations  beside  the  U.S.  They  cooperate 
with  New  Zealand  to  attain  rational 
production  quantities  for  many  types  of 
defense  material,  and  the  United  Kingdom, 
primarily  on  naval  projects.  Australia 
desires  to  strengthen  local  ties.  There  have 
been  successes  in  joint  exercises,  logistics, 
and  sales,  but  no  armaments  cooperation  as 
of  this  writing. 

Australians  cite  the  following  as 
difficulties  in  cooperating  with  the  U.S. 
There  often  seems  to  be  an  issue  on  release 
of  technical  information.  They  complain  of 
being  "ambushed  by  the  many",  a 
reference  to  the  large  number  of  players  in 
the  U.S.  approval  process.  They 
acknowledge  commitment  at  the  working 
level,  but  lacking  in  the  staff  and  financial 
community.  The  "NIS  syndrome"  was 
mentioned.  This  is  an  Australian 
perception  that  if  the  defense  article  is  "not 
in  service"  in  Australia,  then  the  U.S.  is  not 
interested.  Also  mentioned  as  difficulties 
were  the  great  distance  between  the  two 
nations,  the  12-hour  time  difference, 
differing  national  priorities  and  the  size 
mismatch  on  production  rates  and 
quantities. 

Due  to  our  long  history  of  military 
cooperation,  lack  of  economic  competition 
and  common  motivations  for  armaments 
cooperation,  there  exists  few  difficult 
cooperation  issues  from  the  U.S. 
perspective.  Access  to  software  source 
codes  is  an  issue,  but  the  U.S.  historically 
does  not  release  these  to  any  nation. 
Australia  seems  like  a  natural  candidate  for 
expanded  cooperation. 

Japan 

An  understanding  of  the  potential  for 
cooperative  acquisition  projects  with  Japan 
must  begin  with  a  review  and 
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understanding  of  Japanese  policies 
regarding  their  defense  relationship  with 
the  United  States.  These  policies  include 
the  Japanese  "No  War"  Constitution  (post 
World  War  II),  the  Mutual  Defense 
Assistance  Agreement  (1954),  the  Japan- 
U.S.  Security  Treaty  (I960),  the  Three 
Principles  on  Arms  Export  (1967), 
Government  Policy  Guidelines  on  Arms 
Export  (1976)  and  the  Agreement  on 
Technology  Exchange  (1983).  Basically 
these  policies  preclude  Japan  from 
exporting  armaments,  and  from  sharing 
defense  technology  with  any  nation  other 
than  the  U.S.  There  is  an  anti-military 
sentiment  within  Japan,  and  to  further 
confound  cooperation  there  is  an  anti- 
Japanese  military  sentiment  in  neighboring 
Far  Eastern  nations.  There  are  deep 
cultural  differences  between  us.  The 
economic  difficulties  between  the  U.S.  and 
Japan  are  reported  almost  daily  in  the 
American  press.  In  summary,  many 
external  factors  hinder  the  formation  of 
cooperative  acquisition  projects  with 
Japan. 

The  Japanese  Defense  Agency  (JDA) 
conducts  very  little  in-home  research,  but 
cooperation  in  research  is  the  most  feasible 
area.  This  is  because  the  JDA  does  not 
purchase  unlimited  rights  to  intellectual 
property  associated  with  defense  articles, 
unlike  the  U.S.  practice.  The  Japanese 
favor  classified  agreements  which  further 
complicates  cooperation.  The  Japanese 
examine  the  possibility  of  cooperation 
based  upon  four  "merits".  These  merits 
address  the  following  areas: 

Appropriate  for  the  Japanese 
environment. 

Improvements  after  procurement 
possible  using  Japanese 
technology. 

Long  term  logistics  support 
available. 


Enhances  the  growth  of  the 
Japanese  defense  industrial  base 
and  technology. 

While  Japanese  indigenous  RDT&E  is  of 
paramount  importance,  the  Japanese  view 
some  cooperation  with  the  U.S.  as 
"necessary."  Japan  responds  to  U.S. 
initiatives  in  cooperation,  seldom  if  ever 
initiating  cooperative  acquisition  projects. 

Issues  which  may  arise  in  cooperation 
with  Japan  include  technology  transfer  and 
control  (especially  software),  differing 
capability  of  two  defense  industrial  bases, 
joint  ownership  of  intellectual  property 
rights,  and  technology  flowback.  The  last 
issue  has  been  persistent  due  to 
disagreement  over  the  meaning  of  native 
Japanese  technology  and  the  requirements 
to  provide  this,  or  flow  this  back,  to  the 
U.S. 

Real  cooperation  is  only  possible  with  the 
U.S.  Japan  favors  the  Data  Exchange 
Agreements  and  the  Systems  &  Technology 
Forum  for  identifying  cooperative 
opportunities.  The  future  of  cooperative 
acquisition  projects  will  be  on  a  case-by- 
case  basis,  with  clear  and  complementary 
motivations  often  lacking. 

South  Korea 

Recent  moves  toward  democracy  in  South 
Korea  have  reduced  the  influence  of  the 
military.  However,  defense  industry  still 
responds  to  government  direction.  High 
technology  transfers  to  South  Korea  are 
considered  in  the  context  of  potential 
conflict  or  reunification  with  North  Korea. 
South  Korea  does  little  pure  research,  and 
therefore  favors  coproduction.  All 
cooperative  projects  must  have  application. 

As  with  Japan,  there  are  deep  cultural 
differences  between  the  U.S.  and  Korea. 
To  an  American,  Korean  progress  from 
point  A  to  point  B  is  never  a  straight  line. 
Anticipate  the  Koreans  to  pay  great 


attention  to  detail,  and  to  put  almost 
everything  in  writing.  Saving  face  and 
avoiding  fault  are  vitally  important  to  the 
South  Koreans.  Cooperation  with  the 
Koreans  can  be  personality  dependent. 
Anticipate  changes  to  the  project  with 
changes  in  key  personnel.  South  Koreans 
place  emphasis  on  social  activities,  often  at 
the  expense  of  administrative  support. 
Anticpate  the  need  to  provide 
administrative  support,  even  in  the 
translation  of  English  to  Korean.  Also 
anticipate  a  strong  empahasis  on  adhering 
to  schedules. 

South  Koreans  view  cooperative  projects 
with  the  U.S.  as  easy  to  start,  but  difficult 
to  continue.  They  also  view  the  U.S.  as 
reluctant  to  make  cooperative  projects  with 
South  Korea  work.  Therefore,  they  speak 
of  "turning  our  eyes,"  a  euphemism  for 
more  government  and  industrial  defense 
cooperation  with  other  nations,  primarily 
France  and  Germany.  However  they  still 
claim  to  be  actively  seeking  cooperation 
with  the  U.S. 

The  issues  which  typically  arise  in  U.S.- 
South  Korean  cooperative  projects  include 
technology  transfer  and  control,  third  party 
sales,  intellectual  property  rights,  total 
project  cost  and  Korean  cost  share,  and 
the  transfer  of  research  work  to  a  defined 
project.  The  Koreans  favor  Data  Exchange 
Agreements  and  the  Engineer  Scientist 
Exchange  Program  for  identifying 
cooperative  projects. 

INTERNATIONAL  ACQUISITION 
TOPICS 

The  study  yielded  information  on  the 
following  list  of  acquisition  related  topics, 
each  of  which  is  explored  in  detail  in  this 
section. 

*  Project  profiles 

*  Project  office  profiles 

*  Project  initiation 

*  Program  rationale 


*  Barriers  to  cooperation 

*  Potential  partners 

*  Facilitators  of  cooperation 

*  International  concerns 

*  Requirements/Goals 

*  Equitability 

*  Acquisition  concerns 

*  International  training 

Project  Profiles 

All  the  projects  in  the  PACRIM  study 
were  research  and  development  projects,  at 
least  half  of  which  also  included  some 
significant  test  and  evaluation.  These  were 
relatively  small  R&D  efforts  of  about  $10- 
$15  million,  only  half  of  which  had 
procurement  potential.  These  were 
primarily  technology 

demonstration/insertion  projects,  or 
technical  data  gathering  projects.  There 
were  two  exceptions.  While  the  Japanese 
Next  Generation  Support  Fighter  (FX-X) 
Project  is  a  major  acquisition  program  for 
Japan,  it  is  a  very  small  effort  in  the  U.S. 
consisting  primarily  of  monitoring  the 
technology  flowback.  The  other  exception 
is  the  U.S./Australian  Nulka  project  or  the 
MK-53  Off-Board  Active  Decoy  as  it  is  now 
called.  The  intent  is  for  this  project  to  go 
through  development  and  into  production. 
Almost  all  the  projects  are  of  moderate  to 
high  technical  risk,  as  might  be  expected  in 
early  R&D.  Commercial  spin-off  was 
viewed  as  a  possibility  in  half  the  projects. 

Project  Office  Profiles 

While  fully  integrated  international 
program  offices  with  oversight  and 
guidance  provided  by  an  international 
steering  group  is  favored  in  NATO 
cooperative  acquisition  projects,  this  was 
not  necessarily  true  of  PACRIM  projects. 
The  favored  approach  in  over  half  the 
projects  was  a  dual  project  office  structure, 
where  funds  and  technical  effort  were 
managed  in  each  nation,  with  a  sharing  of 
technology  and  results  regularly  during  the 
conduct  of  the  project.  The  lead  nation 
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approach  was  the  second  most  favored 
approached  in  almost  a  third  of  the 
projects.  There  were  no  integrated 
international  project  management  offices  in 
any  of  the  PACRIM  projects.  This  could 
be  attributed  to  any  or  all  of  the  following: 
lack  of  project  maturity,  bilateral  nature, 
and  the  stipulation  attached  to  Cooperative 
Research  &  Development  funds  (aka  Nunn 
Amendment  funds)  that  the  U.S.  portion  be 
spent  in  the  U.S.  While  there  were  no 
integrated  international  project  offices, 
three  approaches  were  employed  to 
facilitate  the  international  nature  of  the 
projects. 

1.  Liaison  officer:  This  was  used 
with  the  Japanese  FS-X  project  and 
the  Australian  Nulka  project,  the 
latter  having  an  Australian  liaison 
officer  in  the  U.S.  project  office,  with 
no  reciprocity. 

2.  In-Country  support:  This  method 
was  favored  with  the  Korean  projects, 
with  the  support  provided  by  the 
Joint  U.S.  Military  Advisory  Group. 

3.  Embassy  Contact:  This  was  the 
approach  clearly  favored  with  the 
Australian  projects,  where  their 
embassy  in  Washington,  D.C.  plays  an 
active  role. 

The  use  of  an  international  steering 
group,  so  highly  favored  in  NATO  projects, 
was  used  in  only  half  the  PACRIM 
projects.  While  those  who  utilized  a 
steering  group  believed  this  structure 
beneficial,  the  other  half  believed  that  a 
steering  group  was  not  necessary. 

Project  Initiation 

The  study  addressed  the  mechanism, 
rationale,  barriers,  and  facilitator  for 
program  initiation,  as  well  as  an  assessment 
of  international  partner  potential. 
Regarding  the  mechanism  for  program 
initiation,  the  surprising  finding  was  that 


there  was  no  common  approach  or  forum 
for  this.  As  Table  4.  illustrates,  each 
project  began  differently.  While  only  one 
project  initiation  was  attributable  to  a  data 
exchange  agreement,  half  the  project  offices 
mentioned  that  an  existing  agreement 
greatly  facilitated  the  project. 

Program  Initiation  Mechanisms: 

All  Are  Different 


*  Defense  Security  Assistance  Agency 

Initiative 

*  Bilateral  Forum 

*  Office  of  Secretary  of  Defense  Directed 

(To  solve  technical  problem) 

*  Data  Exchange  Agreement 

*  Senior  Level  Bilateral  Meeting 

*  Multilateral  Forum 

-  5  Nations  Meeting 

-  ABCA  Forum 

Table  4. 

Program  Rationale 

Examination  of  the  motivation  of  U.S. 
project  personnel  to  enter  into 
international  acquisition  projects  could 
help  to  identify  future  candidates  for 
cooperation.  Not  surprisingly  over  half 
stated  that  a  common  threat  or  need  was 
the  motivation.  While  this  is  the  expected 
answer,  almost  half  had  another 
motivation.  A  big  reason  was  to  access 
cooperative  R&D  funds.  Other  rationales 
were  political  motivation,  technical  benefit 
and  standardization  goals. 

Barriers  to  Cooperation 

The  U.S.  view  on  barriers  was  very  clear. 
All  the  projects  but  one  identified  the 
cumbersome  MOU/MOA  process  as  a 
barrier  to  cooperation.  The  specific 
problem  or  complaint  took  many  forms  and 
involved  such  things  as  the  difficulty  of 
staffing  the  MOU/MOA,  the  length  of  time 
associated  with  the  process  (almost  always 
significantly  underestimated),  and  the 
difficulties  associated  with  a  change  of  legal 
advisor,  reopening  a  MOU  to  negotiation, 
and  the  use  of  a  programme  MOU  (for  the 
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entire  R&D  and  production  cycle)  vice  an 
MOU  for  a  single  phase  of  acquisition. 
Only  one  other  barrier  surfaced.  In  half 
the  projects,  objections  from  other  agencies 
or  departments  were  identified  as  a 
problem.  Mentioned  in  order  of  frequency 
were  the  Defense  Technology  Security 
Administration  (DTSA),  the  Departments 
of  Commerce  and  State,  and  finally  other 
Military  Departments. 

The  allied  view  identified  cumbersome 
U.S.  procedures  as  the  major  barrier  to 
cooperation.  While  the  MOU  process  was 
mentioned  most  frequently,  the  allies  also 
mentioned  encountering  difficulties  with 
our  testing  and  technology  release 
procedures.  Surprisingly  almost  half  the 
PACRIM  allies'  project  offices  cited 
national  budget  processes  being  out  of 
phase  as  a  barrier,  while  U.S.  project 
offices  never  mentioned  this  as  a  barrier. 
U.S.  project  personnel  need  to  be  more 
sensitive  to  our  allies'  differing  budget 
cycles. 

Facilitators  of  Cooperation 

Not  surprisingly,  over  half  of  the  U.S. 
project  offices  representatives  focused  on 
the  project  requirements.  The  term 
"requirements"  could  refer  to  a  technical 
objective,  operational  requirement, 
specification  or  numbers  of  production 
units,  and  was  not  defined.  Nevertheless, 
clarity,  stability  and  mutual  understanding 
of  project  requirements  were  considered  to 
be  of  paramount  importance.  Also  cited 
with  nearly  the  same  frequency  was  high 
level  commitment  and  support  to  the 
cooperative  project.  Specific  examples  (ie. 
Ducted  Rocket  Engine)  were  mentioned 
that  without  OSD  support  the  project 
would  have  never  moved  forward.  Other 
facilitators  cited  were  a  perception  of 
equitability  of  benefits,  having  an  in¬ 
country  liaison,  and  common  program 
objectives.  An  interesting  note  was  that 
only  one  U.S.  project  office  stated  that  they 
had  no  significant  problems  during  the 


MOU  process.  They  gave  the  reason  that 
they  had  engaged  in  two  years  of 
preplanning  and  technical  discussion  under 
an  existing  Data  Exchange  Annex  (DEA) 
with  their  allied  counterparts  prior  to 
entering  the  formal  international 
negotiation  process. 

A  comparison  between  U.S.  and  allied 
views  on  facilitator  for  cooperation  was 
most  revealing.  One  of  the  two  most 
common  responses  from  allied  project 
offices  was  having  a  common  goal.  This 
was  similar  to  the  U.S.  view  but  seemed 
oriented  toward  broader  program  goals, 
rather  than  the  specifics  of  technical  or 
operational  requirements.  The  other  most 
common  response  was  "trust".  This  was  a 
surprise  in  that  it  was  the  number  one 
allied  answer,  but  was  never  mentioned  by 
U.S.  project  office  representatives.  This 
same  phenomenon  occurred  during  studies 
of  European  projects5,1  where  the  need  for 
"commitment"  was  mentioned  often,  but 
exclusively  by  the  Europeans.  This  suggests 
a  profound  cultural  difference  between  the 
U.S.  acquisition  personnel  and  their  allied 
counterparts  regarding  the  value  placed 
upon  trust  and  commitment  necessary  in 
an  international  project.  Two  additional 
answers  were  mentioned:  complementary 
skills/technology  and  prior  meetings, 
neither  of  which  appeared  on  the  U.S.  list. 
It  is  essential  to  understand  the  differences 
in  motivation,  and  that  all  differing  sets  of 
motivations  must  be  satisfied  to  attain 
success. 

Potential  Partners 

Here  again  the  differences  between  the 
U.S.  and  allied  views  are  revealing.  When 
asked  about  the  desirable  characteristics  of 
a  potential  international  partner,  U.S. 
project  office  personnel  found  consensus  on 
only  one  answer:  mutual  interest.  No 
other  answer  appeared  more  than  once,  but 
the  list  included:  available  funds,  win-win 
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Farr,  op.cit. 


attitude,  high  level  advocacy,  technical 
capability,  commitment,  a  signed  royalty 
agreement,  or  a  perception  of  urgency. 
One  of  the  two  most  common  allied 
responses  was  common  goal/need,  similar  to 
the  most  prevalent  U.S.  response. 
However,  it  came  from  less  than  a  third  of 
the  project  offices,  and  was  equal  in 
frequency  of  mention  to  complementary 
skills/technology.  The  latter  was  never 
mentioned  by  U.S.  representatives.  Other 
responses  referred  to  the  existence  of  a 
political  alliance,  past  experience,  proven 
performance  and  reliability, 
interoperability  of  defense  equipment,  and 
equal  partner  mentality. 

International  Concerns 

An  assessment  of  five  aspects  of 
international  projects  as  to  their  impact 
upon  Pacific  Rim  projects  was  conducted. 
Both  the  U.S.  and  allied  views  were 
considered. 

Geographic  separation 

The  first  international  aspect  examined 
was  geographic  separation.  Not 
surprisingly  the  U.S.  project  offices  viewed 
this  as  a  problem  with  all  three  of  the 
nations  considered.  What  was  unexpected 
was  that  our  Australian,  Japanese  and 
South  Korean  counterparts  minimized  the 
importance  of  this  aspect.  They  often  cited 
modern  technology  easing  this  problem. 
First  hand  experience  proves  the  necessity 
of  a  24-hour  fax  machine  for  efficient 
communication. 

Cultural  Differences 

The  second  international  aspect  examined 
was  cultural  differences.  The  U.S.  project 
offices  cited  this  as  a  significant  problem 
when  working  with  Japan  and  South 
Korea,  but  of  minimal  concern  with 
Australia.  Not  surprisingly  the  Australians 
agreed  with  the  U.S.  view.  The  Japanese 
saw  cultural  differences  as  a  problem,  but 


not  a  significant  one.  The  surprising 
response  from  the  South  Koreans  was  that 
cultural  differences  between  the  U.S.  and 
South  Korea  were  of  minimal  impact  in 
international  projects. 

Language  Differences 

The  third  international  aspect  examined 
was  language  differences.  Not  at  all 
surprising  was  the  agreement  between  the 
U.S.  and  Australians  that  this  was  a 
minimal  impact  on  an  international 
project.  However,  a  few  Australians 
mentioned  that  subtle  differences  in 
meaning  and  pronunciation  occasionally 
caused  a  problem.  A  personal  experience  at 
the  Sydney  airport  illustrates  this  point.  It 
took  me  three  times  to  get  a  cup  of  coffee 
because  the  lady  behind  the  counter  kept 
asking  me  "what?"  after  each  request.  As 
it  turns  out  she  was  really  asking  me 
whether  I  wanted  my  coffee  with  milk,  or 
coffee  "white"  as  it  is  commonly  called  in 
Australia.  The  U.S.  project  offices  agreed 
with  their  Japanese  counterparts  that 
language  was  a  problem,  but  not 
significant.  It  was  stated  that  most  of  the 
Japanese  we  deal  with  in  cooperative 
defense  acquisition  projects  were  educated 
in  the  U.S.  and  could  read  English  very 
well.  There  was  some  difficulty  cited  in  the 
spoken  English.  Regarding  the  South 
Koreans,  the  U.S.  saw  language  as  a 
significant  barrier  to  cooperation,  while  the 
Koreans  saw  it  as  a  lesser  problem.  This 
was  expressed  by  one  Korean  project 
officer:  "Language  differences  are  not  a 
great  problem  because  we  speak  the 
common  language  of  science."  Like  their 
Japanese  counterparts,  many  South 
Korean  scientists  obtain  some  of  their 
education  in  the  U.S. 

Technical  Capability 

The  fourth  international  aspect  examined 
was  technical  capability,  especially  whether 
differences  in  technical  capability  between 
the  U.S.  and  partner  nations  caused 
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significant  problems.  There  was  consensus 
between  the  U.S.  and  allied  project  offices 
that  this  was  a  problem,  but  not  a 
significant  one.  There  were  two  exceptions. 
The  South  Koreans  did  not  believe  this  to 
be  a  problem.  The  U.S.  project  offices 
believed  that  this  was  not  a  problem  with 
Japan  at  the  technology  level,  but  that  the 
integration  of  technologies,  components 
and  subsystems  into  a  major  defense  system 
could  be  a  concern. 

Managerial  Differences 

The  fifth  and  final  international  aspect 
examined  was  managerial  differences. 
There  was  clear  consensus  between  U.S. 
and  allied  project  offices  that  this  was  a 
problem  area.  Not  a  single  nation  indicated 
that  this  was  of  minimal  impact.  There 
were  varying  degrees  of  concern  by 
country.  The  U.S.  and  Japanese  project 
offices  agreed  that  this  was  a  significant 
problem  area.  The  U.S.  project  offices 
believed  this  also  was  a  significant  problem 
with  the  South  Koreans.  The  South 
Koreans  did  not  believe  it  to  be  as 
significant  as  the  U.S.  project  offices.  The 
U.S.  and  Australian  project  offices  both 
believed  this  to  be  a  problem  area,  but  not 
as  significant  as  with  Japan. 
Unfortunately,  the  study  did  not  get  into 
the  specifics  of  the  managerial  differences 
that  caused  problems,  and  is  an  area 
worthy  of  additional  research.  In 
summary,  managerial  differences  appear  to 
be  the  greatest  international  concern  in 
intentional  cooperative  acquisition  projects 
with  PACRIM  nations  (especially  with 
Japan),  differing  technical  capability 
caused  some  concerns  and  the  geographic 
separation,  cultural  differences,  and 
language  differences  seem  to  be  of  lesser 
impact.  However,  in  general  the  U.S. 
project  personnel  viewed  the  international 
concerns  as  more  significant  barriers  than 
their  allied  counterparts.  Most  of  our  allies 
are  used  to  obtaining  defense  equipment 
from  outside  their  own  country,  while  the 
U.S.  is  accustomed  to  purchasing 


domestically  produced  defense  equipment. 

Requirements/Goals 

An  assessment  of  the  project 
requirements/goals  process  was  conducted. 
This  covered  technical  requirements, 
operational  requirements,  and/or  general 
project  goals  agreed  upon  by  the  nations 
involved.  This  was  found  to  be  the  most 
troublesome  aspect  of  the  NATO-Europe 
projects.  However,  with  PACRIM 
projects,  we  either  learned  some  lessons 
from  the  NATO-Europe  projects,  or  it  is 
just  too  early  in  the  acquisition  cycle  to 
detect  problems  with  the 
requirements/goals.  The  findings  were  that 
the  requirements/goals  were  jointly 
developed,  specified  at  the  onset  of  the 
project,  user  needs  were  apparent,  and 
there  were  no  significant  problems.  There 
was  one  significant  problem  with  the 
development  of  the  requirements/goals. 
Political  pressure  altered  the  goals  in  half  of 
the  programs,  but  only  before  the  MOU 
was  signed.  After  the  exposure  to  the 
formal  MOU  process,  the 
requirements/goals  stabilized.  This  points 
to  a  strong  need  for  the  acquisition 
manager  to  minimize  the  exposure  time  of 
the  project  to  the  formal  MOU/MOA 
process. 

Equitability 

Since  there  is  a  statutory  requirement 
that  international  cooperative  programs  be 
"equitable",  U.S.  project  offices  were 
surveyed  as  to  their  opinion  on  the  benefits 
received  by  the  U.S.  and  their  perception  of 
the  benefits  received  by  their  partners  in 
the  project.  As  Figure  1.  shows,  the 
benefits  were  perceived  to  be  moderate  to 
significant  Clearly  the  U.S.  project  offices 
perceived  equitable  benefits  from  the 
cooperation.  Partner  exploitation  did  not 
appear  to  be  a  problem  as  about  three 
quarters  of  U.S.  project  offices  believed 
that  neither  partner  was  exploited  and  the 
remainder  said  it  was  too  early  to  tell. 
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Figure  1.  Equitability 


Acquisition  Concerns 

A  general  assessment  was  conducted  of 
the  key  areas  of  acquisition  uncertainty 
associated  with  these  international  projects. 
This  is  shown  in  Figure  2.  Especially 
pronounced  were  the  high  percentage  of 
projects  experiencing  impacts  on  cost  and 
schedule.  The  areas  of  uncertainty  are 
clearly  identifiable  and  appear  significantly 
frequent.  Regrettably,  there  is  no 
comparable  set  of  data  for  domestic 
acquisition  projects,  which  might  show 
similar  problems. 


Figure  2.  Acquisition  Concerns 


International  Training 

An  assessment  of  the  international 
training  needs  of  the  U.S.  project  offices 
was  conducted.  Fully  two  thirds  said  that 
it  was  needed,  and  would  have  helped  the 
early  stages  of  the  project.  The  topics  most 
frequently  mentioned  were  international 
agreements  and  intellectual  property  rights. 
Also  mentioned  were  third  party  transfers 
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beyond  the  nation  participating  in  the 
project,  cultural  aspects,  foreign  policy,  and 
Foreign  Military  Sales  (FMS). 

General  Comparisons  Between  NATO- 
Europe  and  Pacific  Rim  Projects 

Table  5.  lists  some  general  comparisons 
between  the  NATO-Europe  projects  from 
previous  studies  and  the  Pacific  Rim 
projects  examined  under  this  study.  The 
key  point  is  that  the  cooperative  projects 
from  the  two  regions  are  considerably 
different.  I  conclude  that  anyone  well 
versed  in  cooperative  projects  with 
European  nations  may  need  to  relearn  the 
business  when  working  with  the  Pacific 
Rim  nations. 


Table  5. 


FINAL  THOUGHTS 


Anticipate  Problems 


PACRIM  Cooperation  Is  Different  True 
cooperation  in  acquisition  projects  is  in 
its  infancy  in  the  Pacific  Rim.  It  is  twenty 
years  behind  our  efforts  with  the  NATO- 
Europe  nations.  Experience  with  NATO 
projects  may  be  of  little  value  in  the 
Pacific  Rim.  Has  the  U.S.  defense 
acquisition  community  learned  the  Rx  for 
success?  On  the  one  hand  we  seem  less 
prone  to  cancelling  international  projects; 
on  the  other  hand,  the  projects  have 
degraded  to  rather  simple,  early  R&D 
efforts.  Each  nation  merits  a  special 
remark.  Japan  is  unique.  Japan  is  most 
difficult  to  work  with  because  of 
managerial  differences  and  their  pacifist 
policies.  However,  Japan  is 

technologically  mature,  and  therefore 
offers  the  potential  for  significant  mutual 
benefit  from  cooperation.  The 

conundrum  with  cooperation  with  Japan 
in  acquisition  is  that  it  is  simultaneously 
politically  driven  and  politically  opposed. 
Cooperation  with  South  Korea  will  be 
clouded  for  the  near  future  due  to  our 
difficulties  with  North  Korea  and  the 
uncertainties  associated  with 
reunification.  With  respect  to 

cooperation  in  acquisition  with  Australia, 
it  is  difficult  to  understand  why  there  is 
not  more.  While  some  difficulties  exist,  I 
could  find  no  clear  reason  for  the  minimal 
amount  of  cooperative  projects. 

Expectations  for  cooperative  projects 
with  PACRIM  nations  should  be  realistic. 
The  key  to  future  success  will  be  to 
demonstrate  commitment  and  build  trust. 
As  a  final  thought  on  the  different  nature  of 
cooperation  in  the  Pacific  Rim,  is  that  I 
believe  that  the  multiple  bilateral  approach 
may  not  provide  sufficient  synergy  for 
continued  cooperation.  Is  there  a  forum  in 
which  all  the  nations  could  participate  to 
increase  cooperation? 


There  are  key  problem  areas  that  the 
acquisition  manager  should  anticipate 
when  entering  into  a  cooperative  project  in 
the  Pacific  Rim.  These  are  the  known 
unknowns;  there  are  no  clear  solutions,  but 
certain  strategies  can  mitigate  the  impacts. 
First  and  foremost,  anticipate  significant 
problems  during  the  formal  international 
agreement  (MOU/MOA)  process. 
Anticipate  this  to  result  in  changes  to 
project  objectives,  schedule  delays  and 
funding  problems.  These  agreements 
normally  are  signed  at  very  high  levels, 
sometimes  as  high  as  the  Secretary  of 
Defense,  and  seldom  lower  than  the  Defense 
Under  Secretary  or  Service  Secretary  level. 
This  causes  many  organizations  to 
participate  in  the  process,  many  of  which 
have  conflicting  agendas.  The  strategy  is  to 
minimize  your  exposure  time  to  the  political 
levels  encountered  during  the  MOU/MOA 
process.  Some  of  these  problems  may  be 
reduced  with  the  recent  streamlining  policy 
promulgated  by  the  Deputy  Secretary  of 
Defense.6 

It  is  imperative  to  resolve  as  many  issues 
as  possible  before  starting  to  negotiate 
formally.  Many  technical  points  can  be 
resolved  under  a  DEA.  The  main 
restriction  is  that  a  draft  MOU/MOA,  or 
similar  document  cannot  be  tabled  or  even 
discussed.7  I  advise  all  acquisition 
managers  even  contemplating  an 
international  cooperative  project  to  get  a 
DEA  in  place  as  quickly  as  possible  and  use 
it  as  a  vehicle  to  resolve  as  many  issues  as 


Memorandum  from  the 
Deputy  Secretary  of  Defense, 
dated  14  September  1994, 
Subject:  Streamlining  the 
Development  of  International 
Research  and  Development  (R&D) 
Agreements . 

7  DoD  5530.3, 
International  Agreements,  June 
11,  1987. 
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possible  prior  to  formal  negotiation.  In 
other  words,  minimize  your  exposure  time 
to  the  highly  political  MOU/MOA  process. 

Second  and  nearly  as  troublesome, 
anticipate  objections  to  your  international 
project.  These  objections  can  come  from 
virtually  anywhere  to  include  other 
Military  Departments,  other  DoD  agencies, 
and  other  government  agencies  (including 
State,  Commerce,  and  possibly  Treasury), 
as  well  as  the  Congress. 

The  international  acquisition  manager  is 
a  consensus  builder  dealing  with  a  plethora 
of  nay  sayers  far  exceeding  that  found  in 
domestic  program.  Begin  coordination 
early  to  build  consensus.  Advocacy  for 
your  international  project  within  the  OSD 
and  services  international  programs  staffs 
is  essential.  With  this  it  will  be  difficult; 
without  this  impossible. 

A  List  of  Do's 

While  international  cooperative 
acquisition  projects  are  fraught  with 
pitfalls,  they  can  be  successful.  In  fact  all 
the  projects  studied  were  considered 
successful,  and  most  or  all  will  successfully 
meet  their  original  goals.  Based  upon  this 
study,  and  the  years  of  research  preceding 
it,  I  proffer  the  following  "list  of  do's." 

1.  Concentrate  on  mutual  benefits  and 
needs. 

Always  try  to  assess  your  allies'  needs, 
and  arrange  for  equitable  benefits.  While 
the  equitability  of  the  project  from  a 
strictly  U.S.  view  will  be  determined  by 
many  sources,  no  one  in  the  U.S.  is 
responsible  for  looking  at  our  partner's 
needs.  The  acquisition  manager  is 
normally  the  one  who  must  live  with  the 
agreement  and  execute  the  international 
project.  He  or  she  should  strive  for  a 
win-win  situation,  if  additional 
international  activities  are  envisioned. 


2.  Take  fresh,  creative  approaches. 

International  projects  add  a  layer  of 
complexity  to  an  already  difficult 
acquisition  process  that  does  not  readily 
accommodate  international  projects. 
Recent  initiatives  and  innovations  may 
smooth  out  some  of  the  difficulties.  Some 
examples  are  the  recent  streamlining  of 
the  international  agreements  process,  the 
use  of  special  types  of  agreements  (such  as 
the  umbrella  and  chapeau  agreements)  a 
computer  program  to  assist  agreements 
negotiators,  and  the  use  of  DEAs  to 
resolve  early  issues.  Surely,  more 
creativity  will  be  needed  during  the 
project  execution  phases,  beyond  just  the 
approval  of  the  MOU/MOA. 

3.  Stabilize  and  clarify  requirements. 

While  requirements  did  not  appear  as 
problematic  as  in  the  past  studies  of  a 
NATO-European  projects,  it  is  too  soon  to 
tell  whether  this  will  become  the  show- 
stopper  as  some  of  the  projects  progress 
through  the  acquisition  cycle.  Nevertheless, 
even  at  these  early  stages,  stable,  clear 
requirements  were  well  recognized  as  the 
primary  facilitator  of  a  cooperative  project, 
especially  by  the  U.S.  project  personnel. 

4.  Prepare  and  coordinate  up  front 
and  early. 

Consensus  and  advocacy  are  essential 
elements  in  all  acquisition  projects,  but 
the  level  and  span  for  international 
projects  is  much  greater  and  extends 
beyond  just  the  DoD. 

5.  Minimize  exposure  time  to  the 
MOU/MOA  process. 

Exposure  to  the  political  levels  during  the 
formal  negotiation  process  is  likely  to 
result  in  changes  to  project  goals,  as  well 
as  significant  impacts  on  schedules  and 
funding.  The  acquisition  manager  must 
exert  every  effort  to  shorten  exposure 


532 


time  by  building  consensus  and  resolving 
issues  before  the  process  formally  begins. 

6.  Learn  to  be  trustworthy. 

This  was  the  greatest  cultural  divide 
between  Americans  and  our  allies.  Trust 
is  of  great  importance  to  our  Pacific  Rim 
allies,  but  never  mentioned  by  Americans 
as  essential  elements  of  cooperative 
acquisition  projects.  This  seems  to  be 
further  exhibited  in  the  U.S.  approach  of 
addressing  every  possible  contingency  in 
the  extraordinarily  lengthy,  detailed 
project  agreements,  for  which  we  have 
resorted  to  computer  programs  to 
develop. 

7.  Train  and  educate  acquisition 
professionals  before  they  start  the 
international  dialogue. 

It  was  clear  during  the  study  that  none  of 
the  U.S.  project  personnel  had  taken 
advantage  of  available  international 
training.  This  deficiency  has  been 
noticed  and  documented  before.8 
International  projects  require  PET: 
preparation,  experience  and  training. 
The  reality  is  that  U.S.  personnel  often 
pull  the  proverbial  "PET"  rabbit  out  of 
the  hat  when  it  comes  to  international 
projects.  On  October  1,  1994,  all  of 
DSMC's  three  international  acquisition 
courses  were  officially  identified  as 
"assignment-specific  Defense  Acquisition 
University  courses"  by  the  Under 
Secretary  of  Defense  for  Acquisition  and 
Technology.  It  remains  to  be  seen  how 
this  will  be  implemented  within  the 
acquisition  workforce.  The  services  have 
already  expressed  a  desire  to  send  nearly 
10,000  acquisition  workforce  personnel  to 
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our  international  courses.  I  believe  this 
will  bring  about  a  grass  roots  revolution 
in  our  ability  to  engage  in  international 
projects.  The  ultimate  solution  will  be  to 
have  certified  international  acquisition 
corps  personnel  managing  all  of  DoD's 
international  projects  and  related 
activities. 

LIST  OF  ACRONYMS 

ABCA  -  American-British-Canadian- 
Australia  (Refers  to 

the  standardization  agreement  and 
organization  comprised  of  these  countries, 
with  New  Zealand  as  an  observer.) 

ADD  -Agency  for  Defense  Development 
(Refers  to  the  South  Korean  Defense 
Agency  responsible  for  development  of 
defense  equipment.) 

AFB  -  Air  Force  Base 

DAU  -  Defense  Acquisition  University 

DEA  -Data  Exchange  Annex  (Refers  to  an 
annex  on  a  particular  technical  area  to  a 
Master  Data  Exchange  Agreement 
between  the  U.S.  and  another  nation. 
Allows  for  the  international  exchange  of 
scientific  and  technical  information 
among  scientists  and  engineers.) 

DMA  -  Defense  Mapping  Agency 

ESC  -  Electronic  Systems  Command, 
USAF,  Hanscom  Field,  MA. 

FS-X  -  Fighter  Support-Experimental 
(Original  designation  of  the  next- 
generation  Japanese  tactical  fighter.  Now 
designated  as  the  Next  Generation 
Support  Fighter.) 

JDA  -  Japanese  Defense  Agency 
(Equivalent  to  the  U.S.  Department  of 
Defense,  but  is  not  a  cabinet  level 
department.) 
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JORN  -  Jindalee  Operational  Radar 
Network  (Refers  to  a  large  Australian 
project  which  included  the  U.S.-Australian 
Radar  Activities,  or  Over-the-Horizon 
Radar,  Project.) 

JUSMAG-K  Joint  U.S.  Military  Advisory 
Group  -  Korea  (See  also  MDAO  and  ODC 
as  similar  organizations  in  Japan  and 
Australia.) 

MDAO  -  Mutual  Defense  Assistance  Office 
(See  also  ODC  and  JUSMAG-K  as  similar 
organizations  in  Australia  and  South 
Korea.) 

MICOM  -  Missile  Command,  U.S.  Army, 
Huntsville,  AL. 

NRL  -  Naval  Research  Laboratory, 
Washington,  D.C. 

ODC  -  Office  of  Defense  Cooperation  (See 
also  MDAO  and  JUSMAG-K  as  similar 
organization  in  Japan  and  South  Korea.) 

PEO  -  Program  Executive  Officer 

PM  -  Project  Manager 

PMO  -  Project  Management  Office 

PO  -  Project  Office  or  Officer 

SPO  -  Systems  Project  Office 

S&TF  -  Systems  and  Technology  Forum  (A 
bilateral  U.S.-Japan  forum  for  exchanging 
technical  information  and  identifying 
potential  cooperative  projects.) 

TAD  -  Theater  Air  Defense 

TCES  -  Technical  Center  for  Explosive 
Safety,  U.S.  Army,  Savanna,  IL. 

TRDI  -  Technical  Research  &  Development 
Institute,  Tokyo,  Japan  (The  research  and 
development  part  of  the  Japanese  Defense 
Agency.) 


TTCP  -  The  Technical  Cooperation 
Program  (A  program  created  to  acquaint 
participating  countries  -  see  ABCA  -  with 
military  R&D  programs  to  promote 
international  cooperation.) 
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ABSTRACT 

Today's  Navy  faces  an  environment  that 
restricts  personnel  staffing,  distributes 
responsibility  for  supporting  logistics  tasks 
and  yet  demands  extremely  rapid 
modifications  to  logistics  support  and 
operations  support  data.  In  the  Navy’s 
fiscally  conservative  environment,  this  can 
only  that  the  Navy  can  expect  in  the  future, 
this  can  only  be  achieved  by  applying  new 
information  systems  methods  and  tools  to 
access,  update,  and  integrate  the  data  stored 
in  widely  dispersed  organizational 
information  systems.  This  paper  describes 
the  design  and  implementation  of  an 
experimental  database  coordination  and 
workflow  management  system  for  the 
logistics  support  processes  to  serve  in  this 
demanding  environment  for  the  Navy's 
ARLEIGH  BURKE  class  guided  missile 
destroyer. 

Reducing  the  expensive  shore-side 
infrastructure  required  to  support  the 
BURKE  Class  destroyer  while  not 
compromising  warfighting  readiness  of  the 
ARLEIGH  BURKE  class  guided  missile 
destroyers  is  the  target  for  the  AEGIS 
Program  Manager's  DDG  51  ILS 
Improvement  Project.  The  first  portion  of 
this  demonstration  project  has  consisted  of 
the  automation  of  systems  and  coordination 
procedures  to  enhance  the  existing  planned 
maintenance  system  for  the  ship's  machinery 
control  system. 


The  AEGIS  Program  Manager  has  focused 
the  DDG  5 1  Logistics  Support  Improvement 
Project  on  interfacing  six  of  the  most 
important  logistics  support  processes  which 
promise  to  return  the  ''most  bang  for  the 
investment  buck".  Stand-alone  hierarchical 
processes  are  commonly  referred  to  as 
Stovepipes”.  These  processes  are:  the 
Planned  Maintenance  System  (PMS),  the 
Engineering  Change  Proposal  (ECP) 
Process,  the  Allowance  Parts  Lists  (APL's), 
Ship's  Manning  Documents  (SMD's), 
Training,  and  Technical  Manuals.  Each  of 
these  six  organizational  processes  are 
executed  by  thousands  of  Navy  civilian, 
uniformed  and  contractor  support  personnel 
widely  dispersed  across  the  country.  In  the 
past,  these  processes  were  accomplished 
primarily  by  FAX,  paper  based  processes, 
"sneaker  post"  and  the  U.S.  mail. 

The  new  system's  current  major  components 
are  the  product  locator  tool,  a  workflow 
management  tool,  and  the  six  remotely 
distributed  and  independently  maintained 
databases  for  the  support  process.  The 
system  utilized  by  the  Navy  requires  a  logical 
organizational  data  model  and  a  relational 
database  that  allows  for  creative  use  of  the 
relationships  to  identify  and  solve  data 
associated  logistics  support  problems.  The 
goal  was  to  utilize  the  model  to  eliminate 
unsynchronized  entries  into  the  database,  and 
to  avoiding  building  and  storing  complex 
rules  that  must  be  memorized  and  utilized  by 
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sailors.  The  product  locator  contains  a  data 
model  that  integrates  the  data  that  are  used 
in  the  stovepipe  engineering  and 
documentation  system  designs.  In  general,  it 
appears  that  enormous  productivity  benefits 
may  be  derived  from  the  application  of 
integrationand  icreased  systems  connectivity 
available  with  today's  information  system 
technologies.  These  benefits  can  begin  to  be 
tapped  utilizing  the  workflow  management 
and  product  locator  capabilities. 

INTRODUCTION 

The  United  States  Navy,  like  many  other 
large  public  and  private  organizations,  has 
attempted  to  implement  large-scale 
organization  changes.  Among  the  changes 
attempted  are  significant  efforts  to  maintain, 
synchronize  and  integrate  access  to  logistics 
support  databases  that  are  geographically 
widely  distributed  and  administered,  and 
supported  by  separate  functional  command 
and  decision  making  organizations. 

BACKGROUND 

At  the  climax  of  what  was  then  called  the 
"Revolution  at  Sea,"  Vice  Admiral  Joseph 
Metcalf,  a  former  Deputy  Chief  of  Naval 
Operations  for  Surface  Warfare  (OP-03), 
proclaimed  this  goal  with  dramatic 
overtones: 

‘Lead,  follow,  or  get  out  of  the 
way.  This  is  not  a  job  just  for  the 
Admirals.  This  revolution  is  an  all¬ 
hands  working  party,  committed  for 
the  long-pull.” 

His  was  a  quest  for  improvement  and  change 
born  of  personal  conviction  and  vision  that 
only  by  radical  and  comprehensive 
reassessments  of  roles,  missions,  systems, 
technologies  and  operational  practices,  and 


addressing  numerous  organizational 
"religious  issues"  that  worked  to  impede 
clarity  of  thought,  purpose  and  action,  could 
the  Navy's  surface  combatants  of  the  future 
place  maximum  ordnance  on  target.  Since 
the  publication  of  the  Navy-Marine  Corps 
white  paper,  From  the  Sea,  in  early  fall  1992, 
the  Navy  has  labored  to  make  ready  for  the 
21st  Century.  The  Department  of  the  Navy 
is  addressing  strategy,  policy,  doctrine,  force 
structure,  operations,  tactics,  shore-side 
infrastructure,  and  a  myriad  of  other  factors 
from  the  top-down  and  bottom-up  in  what 
some  see  as  a  "no-holds-barred,  nothing-is- 
sacred"  assessment.  Their  ultimate  goal  is  to 
ensure  that  requirements  and  resources  are 
better  aligned,  and  that  missions  are  fulfilled 
without  a  return  to  the  "hollow  Navy"  of  the 
1970s. 

INFORMATION  SYSTEM  CHALLENGES 
FACING  THE  EVOLVING  NAVY 

The  system  development  requirements  of  the 
Navy  have  also  changed  dramatically,  and 
methods  and  tools  must  now  be  selected  to 
fit  the  Navy’s  business  problems  of 
downsizing  and  eliminating  some  tasks 
previously  performed  by  reorganized 
programs  and  departments.  The  Navy  now 
faces  even  more  complex  re-engineering  and 
process  innovation  challenges.  Information 
System  (IS)  requirements  which  must 
influence  changing  business  operations 
represent  a  change  from  previous  Navy 
information  systems  developers  requirements 
to  develop  stand-alone  functional  systems 
with  the  support  of  specific  units  within  the 
Navy. 

Systems  and  projects  to  implement  the 
Navy's  current  requirements  must  be 
composed  of  shared  databases  and  many 
independent  application  systems.  The  role  of 
Navy  IS  development  is  to  now  implement 
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strategic  systems  plans,  and  to  collect,  store, 
and  provide  access  to  the  Navy's  data  when 
it  is  required. 

This  paper  analyzes  the  growing  requirement 
for  new  types  of  systems  development 
methodologies  that  emphasize  the  use  of  new 
Commercial  Off  The  Shelf  (COTS)  tools  in 
combination  with  the  integration  of  a  number 
of  other  technologies  that  are  applied  to 
specific  business  and  organizational 
management  problems. 

NEW  DEVELOPMENT  ENVIRONMENT 

Assessments  of  the  systems  environment 
show  that  the  Navy  is  now  facing  far 
different  information  systems  development 
environments  that  include  vendor  supplied 
tools  and  software,  with  concurrent 
organization  imperatives  to  modify  the 
Navy's  structure  and  operations  to  improve 
efficiency.  New  business  and  operational 
benefits  can  be  realized  by  integrating  today's 
COTS  tools  with  the  other  organizational 
information  systems.  This  is  a  never  ending 
task  with  managers  needing  new  and 
evolving  developmental  techniques.  Navy 
managers  concentrating  on  the  acquisition, 
and  integration  of  COTS  tools  with  strong 
system  integration  and  interfacing 
capabilities  will  find  a  substantial  return  on 
their  investment. 

In  addition,  Navy  managers  must  provide 
software  tools,  control  processes  and 
communication  methodologies  for  indexing, 
accessing,  updating  and  sharing  data  across 
remotely  distributed  databases.  Limitations 
in  methodologies  and  tools  have  prevented 
the  computer  from  becoming  an  important  (if 
not  critical)  ingredient  in  this  process  in  the 
past. 


METHODOLOGIES 

These  new  requirements  have  led  the  Navy 
to  an  understanding  that  a  new  set  of 
environmental,  technical  and  organizational 
methods  and  tools  are  needed  for  computer 
support  and  integration.  New  tools  and 
methods  are  important  for  overall  system 
success,  and  are  far  more  possible  and 
affordable  than  in  the  past. 

It  appears  that  the  previous  typical  system 
development  and  life  cycle  methodologies 
contributed  little  to  the  diagnosis  and 
solution  of  operational  and  integration 
problems  of  this  new  environment.  The 
familiar  software  and  system  methodologies 
included  the  Classic  Life  Cycle  otherwise 
called  the  "waterfall  model"  that  describes  a 
systematic,  sequential  development  approach 
stretching  throughout  the  analysis,  design, 
coding,  testing,  and  maintenance  stages.  The 
prototyping  process  involves  building  a 
model  of  the  system,  customer  evaluation, 
refinement  of  the  requirements,  reworking 
the  model,  further  refinement  of  the 
requirements,  and  eventually  building  the 
desired  system  using  sometimes  portions  of 
the  prototype  (2).  Neither  approach 
addresses  the  complex  integration  and 
implementation  issues  found  today.  These 
problems  require  analysis,  evaluation, 
decisions,  and  tools  that  address  the 
individual  and  specific  integration  concerns 
that  are  not  covered  in  the  traditional 
methodologies. 

A  relatively  new  methodology,  the  Spiral 
model,  seems  to  offers  some  promise  for  the 
implementation  problems  faced  by 
administrative  systems  requiring  significant 
conversion  and  data  transfer.  The  spiral 
methodology  encompasses  features  of  both 
prototyping  and  the  classic  life  cycle.  It 
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includes  four  major  activities:  planning,  risk 
analysis,  engineering,  and  customer 
evaluation.  Progressively  more  complete 
systems  are  built  as  developers  move 
through  the  four  stages  in  a  spiral  (2). 

This  new  methodology  is  required  because 
the  system  criteria  for  assessing  whether  or 
not  logistics  support  requirements  for  ship 
systems  and  equipment  are  effectively  met 
for  the  AEGIS  cruisers  and  destroyers  have 
now  changed  dramatically.  But  the  computer 
support  methodology  and  tools  to  support 
systems  designed  to  integrate  these 
requirements  have  not  kept  up  with  the 
requirements  evolution.  In  addition,  the 
criteria  used  to  assess  the  effectiveness  of 
meeting  requirements  shifted  in  their  relative 
importance.  The  Navy  is  now  using  a  new 
set  of  criteria  that  stress  effective  systems 
integration  as  the  measure  of  success  for 
information  systems  that  attempt  to  address 
an  organization's  needs.  This  article  identifies 
four  new  criteria:  speed,  change  distribution, 
auditability  and  labor  efficiency. 

These  new  system  criteria  address  several 
military  business  problems  associated  with 
the  limitations  of  the  current  Navy 
stovepipe-like  information  systems 
architecture.  First,  the  Navy  must  now 
update  its  separate  logistics  support 
databases  in  a  very  short  period  of  time. 
Therefore,  speed  is  a  critical  factor  in 
assessing  the  overall  success  of  a 
management  information  system  support 
process.  Second,  the  Navy  must  ensure  that 
all  changes  made  to  the  data  are  completely 
propagated  to  the  other  stovepipe  systems 
(databases).  Third,  the  Navy  must  audit  the 
common  and  shared  data  among  the 
stovepipe  systems,  and  assure  that  all  data 
are  correctly  entered,  updated  and  fully 
synchronized.  Finally,  the  Navy’s  logistics 
support  methodology  must  now  be  executed 


in  accordance  with  new  staffing  limitations 
imposed  by  Congressionally  mandated 
downsizing. 

The  Spiral  method  appears  to  be  relatively 
more  successful  because  today's  Navy 
systems  environment  is  facing  restrictions  in 
personnel  staffing  and  distributes  tasks 
among  disparate  command  and  logisitics 
support  organizations.  Yet,  the  Navy  must 
still  be  able  to  rapidly  disseminate 
modifications  and  changes  to  logistics 
support  and  operations  support  data.  Such 
modifications  can  only  be  realized  by  rapidly 
integrating  new  information  systems  tools 
with  the  mechanisms  that  access  and  update 
dispersed  organizational  information 
systems.  This  article  describes  the  design 
and  implementation  of  a  prototype  database 
coordination  and  integration  system  for 
several  of  the  logistics  support  processes  for 
the  Navy's  newest  surface  combatant  -  the 
Arleigh  Burke  class  guided  missile  destroyer. 
This  project  is  in  keeping  with  the  long-term 
goal  of  the  Navy  to  do  more  with  less,  while 
not  sacrificing  the  ability  of  the  United  States 
Navy  to  conduct  prompt  and  sustained 
combat  operations  at  sea  anywhere  in  the 
world.  Most  importantly,  the  methodologies, 
technologies  and  capabilities  emerging  from 
the  project  have  the  potential  to  be  directly 
transitioned  to  any  business  process 
(government  or  the  private  sector),  thereby 
capturing  the  cost  savings  benefits  of 
fundamental  business  process  re-engineering. 

AN  INFORMATION  ARCHITECTURE 

The  Navy  is  moving  toward  an  organization- 
wide  information  architecture,  identified  with 
a  set  of  systems  and  projects  to  implement 
mission  requirements.  This  information 
architecture  is  composed  of  shared  databases 
and  application  systems.  Its  role  is  to 
collect,  store  and  provide  access  to  the 
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Navy's  logistics  support,  training, 
engineering  data  and  other  logistical  support 
data,  as  required.  Databases  and  shared 
applications  must  be  designed  with  common 
business  objectives,  and  development 
projects  must  be  implemented  using  cross 
functional  teams  to  design  the  new  integrated 
systems. 

One  experimental  project  using  this  Spiral 
development  methodology  and  linked  to  the 
Navy's  and  the  Defense  Department's 
Computer-aided  Acquisition  and  Logistics 
Support  (CALS)  effort,  is  being  undertaken 
by  the  Navy's  AEGIS  Program  as  a  method 
to  decrease  weapon  systems  life-cycle  costs 
of  the  Navy's  Arleigh  Burke  (DDG  51)  class 
guided  missile  destroyers.  Embodying  the 
philosophy  of  "build-a-little,  test-a-little, 
learn-a-lot,"  the  DDG  51  class  Integrated 
Logistics  Support  (ILS)  Improvement 
Project  responded  to  VADM  Metcalfs 
challenge.  The  Project  is  now  in  position, 
and  the  Navy  expects  it  to- 

•  evoke  re-engineering  of  fundamental 
logistics  support  business  processes 
within  the  AEGIS  Program;  and 

•  be  the  catalyst  for  fundamental  business 
process  re-engineering  for  the  way  the 
Navy  manages  technical  documentation, 
trains  sailors  and  maintains  complex 
weapons  systems  in  the  21st  Century. 

The  AEGIS  Program's  raison  d'etre  is  the 
design,  construction,  outfitting  and  delivery 
of  AEGIS  warships  to  the  fleet,  and  the 
planning  and  execution  of  modernization  and 
lifetime  support  for  those  ships,  i.e.,  "total 
support,  from  cradle  to  grave."  Indeed,  from 
its  inception  more  than  two  decades  ago,  this 
Surface  Navy  program  always  embraced 
three  key  commitments: 


•  to  deliver  the  most  modem,  affordable, 
capable  and  war-ready  ships  in  the  world; 

•  to  provide  the  best  and  most  affordably 
trained  officers  and  sailors  to  man  those 
ships;  and 

•  to  maintain  the  readiness  and 

modernization  of  those  ships  at  the 
highest  affordable  state  possible 

throughout  their  service  lives. 

Today,  27  Ticonderoga  class  AEGIS  guided 
missile  cruisers  are  in  commission.  Seven 
Arleigh  Burke  class  AEGIS  guided  missile 
destroyers  have  been  delivered,  with  another 
22  either  under  construction  or  under 
contract.  A  total  of  58  destroyers  are 
planned. 

REDUCING  COSTS  BY 
RECAPITALIZING  THE  NAVY 


A  critical  underlying  factor  associated  with 
each  criterion  of  the  Navy's  program  to 
ready  the  Naval  Service  for  the  next 
millennium  is  the  Navy's  requirement  to 
reduce  shore-side  and  other  support 
infrastructure  costs.  This  will  allow  the 
Navy  to  "re-capitalize"  itself  and  maintain  a 
sufficient  force  structure  of  trained  and 
motivated  people  to  meet  the  needs  of  the 
future.  Additionally,  reducing  support 
infrastructure  costs  will  allow  "right-sizing" 
the  fleet  for  the  Navy's  continuing  extensive 
political-military  commitments  within  the 
very  real  and  austere  fiscal  constraints  of 
today. 

Reducing  the  expensive  shore-side 
infrastructure  required  to  support  the  Burke 
class  destroyer  is  one  objective  of  the 
AEGIS  Program  Manager's  DDG  51  ILS 
Improvement  Project.  The  solution  lies  in 
the  capability  of  today's  Information 
Technology  (IT)  and  IT's  ability  to  rapidly 
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integrate  information  horizontally  across  an 
organization. 

TOOL  REQUIREMENTS  FOR  NAVY 
LOGISTICS  SUPPORT  IS  GROWING 

The  Navy  now  faces  complex  re-engineering 
and  process  innovation  challenges.  To  solve 
these  challenges,  the  Navy  expects  its 
information  systems  to  meet  some  or  all  of 
the  different,  and  in  many  cases,  heightened 
criteria  previously  identified.  These  new 
criteria  (speed,  change  distribution, 
auditability,  and  labor  efficiency)  may  appear 
to  alter  the  original  system  requirements  and 
assessment  criteria,  which  previously 
emphasized  data  input,  retrieval,  processing 
or  calculation,  and  production  functionality 
for  predefined  reports  as  the  critical 
measures  of  effectiveness. 

This  change  was  observed  in  other 
information  systems  environments.  For 
example,  Orlikowski  (1)  cites  the  rationale 
for  the  adoption  and  use  of  Computer-Aided 
Software  Engineering  (CASE)  tools  as  being 
in  part  derived  from  information  systems 
managers’  desire  to  implement  a  new 
methodology  and  corporate  architecture  that 
would  facilitate  the  redesign  of  business 
units. 

This  documented  attempt  to  influence 
business  operations  represented  a  change 
from  previous  functions  that  the  information 
systems  group  supported  in  the  organization. 
It  is  an  added  requirement  (or  an  additional 
criteria)  that  may  be  used  to  assess  the 
effectiveness  of  the  information  system  of  the 
organization  studied.  To  meet  this  criteria, 
the  organization  must  no  longer  attempt  to 
develop  stand-alone  functional  information 
systems  with  the  support  of  specific  units 
within  the  organization. 


Orlikowski  cites  specific  examples  where 
organizations  now  modify  their  information 
architecture  and  implement  the  organization's 
requirements  and  evaluation  criteria  with 
new  systems  and  projects.  The  new 
information  architectures  are  composed  of 
shared  databases  and  application  systems. 
The  role  of  information  systems  is  to 
implement  strategic  systems  plans,  and  to 
collect,  store  and  provide  access  to  all  of  the 
organization's  data,  as  required.  Databases 
and  shared  applications  are  funded  through 
common  mechanisms,  and  development 
projects  depend  on  cross  functional  teams  to 
implement  new  integrated  systems  (1). 

Today's  Navy  appears  to  face  a  very  similar 
challenge,  with  a  similar  solution.  The  Navy's 
logistics  support  environment  must 
incorporate  the  support  data  associated  with 
rapid  changes  in  technologies  and  disparate 
equipment/systems  configurations  aboard 
ships  and  aircraft.  Its  support  environment 
must  also  complete  upgrades  in  support 
systems,  logistics  support  data,  training 
requirements  and  technical  documentation. 
These  rapid  changes  can  only  be  realized  by 
using  systems  tools  that  are  integrated  with 
other  organizational  support  systems. 

Navy  managers  must  select  a  mix  of  tools 
and  systems  to  support  all  of  the 
organizational  facets  of  integration  and 
concentrate  on  using  these  tools  to 
coordinate  the  Navy's  diverse  logistics  tasks. 
This  article  describes  how  the  Navy's  AEGIS 
Program  is  defining  new  requirements  for 
logistics  support  systems  and  reevaluating 
criteria.  This  redefinition  of  requirements 
and  reevaluation  of  criteria  is  intended  to 
emphasize  the  use  of  information  systems  as 
organizational  change  tools  for  executing 
business  process  re-engineering  within  the 
logistics  support  processes  for  the  Arleigh 
Burke  class  destroyers. 
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DDG  51  LOGISTICS  SUPPORT 
REQUIREMENTS  AND  EVALUATION 
CRITERIA 

The  Navy's  logistics  support  information 
systems  are  similar  to  systems  found  in  many 
of  today's  corporate  organizations.  The 
systems  combine  many  different  features 
such  as  a  Graphic  User  Interface  (GUI)  for 
front-end  capture  of  information  with  a 
relational  database,  mechanisms  for  file 
storage  and  retrieval,  and  systems 
management  controls  using  a  work  flow 
system.  This  systems  environment  is  typical 
of  organizations  that  are  experiencing 
ongoing  evolution  in  their  information 
systems. 

In  general,  these  systems  must  store  logistics 
data  and  supporting  information  in  an 
organization  memory  that  can  be  used  within 
the  organization.  The  organization  memory 
will  serve  as  a  design  and  development  data 
dictionary  to  support  the  ability  to  use  the 
data  model  as  a  tool  that  can  be  used  to 
demonstrate  the  characteristics  of  the 
business  relationships  among  the  business 
units  in  the  organization.  The  data  model 
must  be  compatible  with  systems  design  and 
in  implementation  to  show  how  departments 
and  units  are  related  to  each  other.  The 
tools  that  support  these  systems  must  link  to 
the  front-end,  PC-based  development 
environment  of  the  organization,  prototype- 
evolving  applications,  and  integrate  the 
various  databases  of  the  organization. 

The  AEGIS  Program  Manager,  in  the  life- 
cycle  support  role  for  AEGIS  class  ships, 
funds  the  AEGIS  Program's  share  of  the 
Navy's  enormous  "stovepipe"  logistics 
support  organization  for  logistics  support  of 
AEGIS  ships.  Today,  the  Navy's  logistics 
support  infrastructure  consists  of  separate 


and  distinct  organizational  processes.  In  only 
rare  instances  (usually  a  crisis  or  other 
emergent  situation  that  forces  manual 
integration)  do  these  processes  interact  with 
one  another. 

The  AEGIS  Program  Manager  focused  the 
DDG  51  Logistics  Support  Improvement 
Project  on  horizontally  interfacing 
information  across  six  of  the  most  important 
logistics  processes  that  promise  to  return  the 
most  for  the  investment  dollar.  These 
processes  are- 

(1)  Planned  Maintenance  System  (PMS). 
This  system  administers  and  accomplishes 
preventive  maintenance  afloat  and  ashore. 

(2)  Engineering  Change  Proposal  (ECP) 
Process.  This  is  the  process  that  reviews, 
approves,  sequences  and  schedules  proposed 
engineering  changes. 

(3)  Allowance  Parts  Lists  (APLs).  These  are 
the  parts  lists  every  ship  maintains,  which 
specify  the  spare  parts  and  quantities  each 
ship  shall  carry. 

(4)  Ship's  Manning  Documents  (SMD). 
These  are  the  documents  that  describe  how  a 
ship  shall  be  manned  in  terms  of  required 
numbers  of  crew  and  their  skill  levels. 

(5)  Training.  This  process  provides  for  the 
correct  training  for  ship's  crews 
commensurate  with  the  types  of  equipment 
and  systems  installed  aboard  the  ship. 

(6)  Technical  Manuals.  This  process 
provides  for  the  production  and  life-cycle 
maintenance  of  the  Navy's  equipment  and 
systems  technical  manuals/documentation. 

Each  of  these  six  organizational  processes 
are  executed  by  thousands  of  Navy 
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uniformed,  civilian  and  contractor  support 
personnel  who  are  widely  dispersed 
geographically  throughout  the  country. 

These  processes  are  accomplished  primarily 
in  a  paper  environment,  and  employ  FAX, 
"sneaker  post"  and  the  U.S.  mail  for 
connectivity.  For  example,  it  could  be 
possible  for  a  PMS  feedback  report 
originated  from  an  Atlantic  Fleet  AEGIS 
ship,  which  recommended  a  change  to  an 
existing  maintenance  procedure,  to  pass 
through  as  many  as  three  separate  shore 
support  organizations: 

(1)  COMNAV SURFLANT  in  Norfolk,  VA; 

(2)  COMNAVSEALANT  in  Norfolk,  VA; 
and  (3)  the  Philadelphia  detachment  of  the 
Naval  Surface  Warfare  Center  -  Carterock 
Division)  before  the  change,  if  approved,  is 
finally  disseminated  to  all  AEGIS  ships. 

Significant  problems  arise  with  this  process  if 
the  new  change  in  maintenance  procedures 
recommended  by  that  Atlantic  Fleet  ship 
drive  other  changes  requiring- 

(1)  additional  training  for  maintenance 
personnel; 

(2)  different  spare  parts; 

(3)  revisions  to  a  technical  manual/operators 
manual;  or 

(4)  a  reduction  or  increase  in  ship's  manning 
requirements. 

It  may  be  many  months  before  all  of  the 
changes  are  entered  into  "the  system" 
correctly,  and  much  longer  before  those 
changes  ever  reach  the  deckplates  aboard  an 
AEGIS  destroyer. 

MAJOR  COMPONENTS  OF  THE 
SYSTEM 

The  basic  objectives  of  the  DDG  51  program 
are  to  use  modeling  and  diagramming 
techniques  to  define  the  data,  processing 


requirements,  and  workflows  of  the  current 
logistics  support  process,  i.e.,  the  ‘hs  is” 
process,  then,  to  employ  Information 
Technology  to  replicate  the  process.  Later, 
close  examination  of  a  model  allows  for 
calling  into  question  each  sub-process  to 
determine  its  value  in  accomplishing  the 
overall  process.  Process  re-engineering  can 
now  begin  using  information  technology  to 
replicate  the  re-engineered  workflow 
processes.  If  this  sounds  a  lot  like  Total 
Quality  Leadership/Total  Quality 
Management,  that’s  because  that’s  exactly 
what  is. 

The  system's  current  major  components  are 
the  product  locator  tool,  a  workflow 
management  tool,  and  the  six  remotely 
distributed  and  maintained  databases.  The 
system  used  by  the  Navy  requires  a  logical 
organizational  data  model  and  a  relational 
database  that  allow  for  creative  use  of  the 
relationships  to  identify  and  solve  data 
association  logistics  support  problems.  The 
goal  was  to  use  the  model  to  eliminate 
unsynchronized  entries  into  databases,  and  to 
avoid  building  and  storing  complex  rules  that 
must  be  memorized  and  used  by  employees. 

The  product  locator  is  a  crucial  portion  of 
the  system  that  provides  the  logical  data 
model.  It  serves  as  the  essential  database  and 
driver  for  the  coordination  of  all  change 
drivers,  and  for  locating  logistics  support 
equipment  data.  It  contains  a  data  model 
that  stores  the  data  that  are  used  in  the 
stovepipe  engineering  and  documentation 
system  designs,  and  may  contain  relationship 
data  that  indicate  how  various 
documentation  and  support  databases  are 
related  to  each  other.  The  system  is  loaded 
with  data  that  provide  the  information  and 
guidance  to  make  integrated  and  coordinated 
changes  in  the- 
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•  Allowance  Parts  Lists; 

•  Technical  Manuals;  and 

•  Navy  Training  Plan  and  Manning 
Documents  Planned  Maintenance  System 
Documentation. 

ENGINEERING  CHANGE  PROPOSAL 
PROCESS 

It  is  important  to  understand  that  the  support 
processes  being  integrated  using  this 
approach  are  widely  dispersed  in  geographic 
locations  and  are  disparate  in  implementing 
technologies.  As  an  example,  the  AEGIS 
Engineering  Change  Proposal  (ECP)  Process 
infrastructure  for  the  Machinery  Control 
System  (MCS)  aboard  the  Burke  class 
destroyers  is  comprised  of  the  prime 
contractor,  Martin  Marietta  Corporation, 
Daytona,  FL;  the  AEGIS  Program's  Change 
Control  Board  (CCB)  in  Washington,  D.C.; 
the  Naval  Sea  Systems  Command  also  in 
Washington,  D.C.,  which  is  the  MCS  life- 
cycle  manager;  the  Philadelphia  detachment 
of  the  Naval  Surface  Warfare  Center  - 
Carterock  Division;  the  Supervisor  of 
Shipbuilding,  Bath,  ME;  and  the  lead 
shipbuilder  of  the  Burke  destroyer,  Bath  Iron 
Works  in  Bath,  ME. 

The  development  of  the  product  locator  by 
the  Navy  is  a  strategic  step  in  the 
implementation  of  newer  information  system 
technologies  to  overcome  the  limitations  of 
the  organization's  old  and  costly,  manual, 
stovepipe-like  environment.  The  new  system 
is  a  conscious  effort  to  modify  the 
uncoordinated  flow  of  information  for 
logistics  support  functions  that  previously 
supported  the  independent  processes  and 
sub-processes. 

Observations  indicate  that  the  stovepipes’ 
support  processes  were  primarily  geared  for 
production  activities,  and  were  not  oriented 


toward  answering  management  questions, 
supporting  analysis  of  business  information, 
or  assuring  that  correct  and  complete 
changes  had  been  made  to  all  support 
documentation  in  all  of  the  other  stovepipe 
systems  for  logistics  support.  The  benefits 
from  the  product  locator  are  particularly 
important  to  the  Navy  in  the  multiple 
database  access  areas.  The  current  stovepipe 
processes  made  it  difficult  to  track  multiple 
sources  of  logistics  support  data  for  one 
piece  of  equipment. 

Logistics  support  changes  made  at  different 
times  or  with  slight  variations  in  the  name  of 
the  equipment  or  part  modified  are  not 
always  available  for  the  sailor  (and  different 
organizations  may  have  updated  the  data  at 
different  points  in  time).  The  impact  of  this 
problem  is  twofold.  The  sailor  is  confused, 
and  the  Navy  must  develop  further  complex 
training  procedures  to  correctly  use  its  rather 
unique  and  uncoordinated  stovepipes  (a 
direct  cost  to  the  organization). 

A  common  product  locator  database  under 
development  by  the  AEGIS  Program 
Manager  for  the  logistics  support  of  the 
Arleigh  Burke  DDG-51  class  destroyer 
solves  the  problem  because  a  single  source 
can  be  examined  to  identify  all  of  the  ship's 
equipment  and  systems  support  data  storage 
locations  maintained  by  the  Navy. 
Associations  among  the  data  can  be 
reviewed.  The  overall  result  is  better 
logistics  support  service,  reduced  training 
and  decreased  cost  to  the  Navy. 

WORK  FLOW  MANAGEMENT  TOOL 

Systems  design  and  development  literature 
emphasizes  the  enormous  productivity  and 
potential  integration  benefits  from  the 
increased  systems  connectivity  available  in 
today's  information  system  technologies. 
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These  benefits  can  begin  to  be  tapped  using 
the  workflow  management  component  of  the 
DDG  51  ILS  system  when  this  system  is 
combined  with  the  product  locator  tool. 

The  work  flow  component  has  the  capability 
of  routing  images  and  files  throughout  the 
logistics  support  organization  network.  It 
may  contain  embedded  timers  to  check  for 
the  occurrence  of  an  event  or  action.  A  file 
may  be  created  in  a  special  directory,  and  a 
process  or  transfer  of  data  initiated  to 
continue  the  logistics  support  processing. 
This  enables  the  work  to  be  monitored  as  it 
passes  through  the  different  functional  areas. 
If  the  files  or  "folders"  are  backed  up  in  one 
area  for  any  reason,  the  work  may  be 
reassigned  to  another  area  for  completion. 
The  workload  balancing  enables  all  of  the 
logistics  support  update  tasks  to  be 
effectively  sequenced  and  scheduled. 

The  work  flow  system  may  also  be  used  to 
establish  mailboxes,  define  processing 
routines,  set  timing  and  triggers  for  the 
execution  of  routines,  establish  processing 
diaries,  maintain  comments,  and  construct 
forms  for  use  in  systems  processing.  It  has 
many  multi-user  capabilities  that  support  the 
integration  required  across  the  different 
functions  performed  in  the  stovepipe 
systems.  This  helps  to  solve  the  enormous 
and  expensive  logistics  support  challenge 
that  the  Navy  faces. 

In  perspective,  the  Navy  is  in  a  crisis  as  to 
how  to  manage  all  this  information.  Until 
now,  the  Navy  was  constrained  to  manage  all 
this  logistics  support  information  using 
paper-oriented,  page-based  manual  processes 
because  the  information  technology  required 
was  either  not  available  or  not  affordable. 

The  Chief  of  Naval  Operations  (Logistics), 
for  example,  estimates  that  the  Navy 


produces  22.5  million  page  changes  to 
technical  manuals  alone  per  year,  using 
approximately  1,000  sailor  man-years  of 
effort  per  year,  and  costing  the  taxpayer 
approximately  $400  million.  For  the  AEGIS 
Program  Manager,  maintaining  technical 
documentation  for  the  sophisticated  and 
highly  complex  AEGIS  Weapon  System 
(AWS)  is  proportionately  just  as  expensive. 
The  Navy  has  already  begun  transitioning 
AWS  technical  documentation  from  paper- 
based  formats  to  highly  interactive,  digital 
format. 

Similar  problems  exist  in  many  other  areas, 
such  as  the  spare  parts  problem  aboard  an 
AEGIS  destroyer.  An  AEGIS  destroyer  has 
approximately  23,000  spare  part  line  items, 
with  each  item  carrying  its  own  distinctive 
part  number.  Each  line  item  consists  of  at 
least  one  spare  part,  and  most  commonly 
there  may  be  carried  more  than  1  unit  of  the 
same  numbered  stock  item.  In  all,  an  AEGIS 
destroyer  typically  carries  six  times  the 
number  of  spare  part  line  items  or 
approximately  138,000  spare  parts. 

To  illustrate  the  enormity  of  the  logistics 
crisis  facing  the  Navy  and  all  of  the 
Uniformed  Services,  and  to  put  this  crisis  in 
the  perspective  of  simply  being  an 
information  management  problem,  imagine  a 
"worst-case"  scenario  where  a  sailor  aboard 
an  Atlantic  fleet  AEGIS  ship  identifies  an 
electronic  circuit  card  problem  in  the  ship's 
Machinery  Control  System  (MCS).  The 
sailor  immediately  and  correctly  submits  a 
feedback  report  on  the  problem,  as  required. 
For  routine,  non-safety  related  feedback 
reports,  it  takes  a  very  long  time  (possibly 
months)  to  alert  all  other  ships  about  the 
problem,  its  resolution,  and  the  materials  and 
instructions  needed  to  fix  the  problem  aboard 
other  AEGIS  ships. 


546 


In  the  interim,  another  sailor  aboard  an 
AEGIS  ship  in  the  Arabian  Sea  encounters 
the  same  problem.  Not  knowing  his  manuals 
and  technical  documentation  are  no  longer 
correct,  the  sailor  unwittingly  attempts  to  fix 
the  first  problem.  Incorrect  tests  are 
performed  and  ultimately,  further  damage 
occurs.  The  sailor  attempts  to  obtain  repair 
circuit  cards  from  supply,  but  finds  that  they 
too  are  incorrect  because  the  APL 
(allowance  parts  list)  is  outdated.  New  parts 
are  requested.  The  correct  parts  arrive  in  2- 
5  days,  but  problems  still  persist  in  the 
system,  and  the  outdated  technical  manuals 
fail  to  cover  these  new  problems. 

Finally,  the  ship  requests  technical  assistance. 
Support  personnel  (technical  assistance  team 
consisting  of  one  or  more  technicians)  are 
flown  from  the  U.S.  to  Saudi  Arabia  and 
transferred  to  the  ship.  Armed  with  the 
correct  technical  information  and  skills,  the 
technical  assistance  team  quickly  restores  the 
MCS  to  full  capability.  Although  this  is  a 
"worst-case"  scenario,  varying  degrees  of 
this  scenario  are  all  too  common, 
tremendously  expensive,  and  caused  simply 
by  inefficient  information  management. 

APPLICATION  OF  THE  SPIRAL 
METHODOLOGY 

Systems  integration  efforts  (using  the  Spiral 
development  model)  involves  iterative 
development  tasks.  In  the  present 
development  phase,  electronically  replicating 
the  existing  logistics  support  processes  for 
the  DDG  51  MCS  is  the  objective.  The 
benefits  here  are  derived  simply  by 
efficiencies  gained  with  eliminating  paper 
roadblocks  and  queues,  and  moving 
information  internally  and  externally 
electronically.  In  this  phase,  no  attempt  was 
made  to  re-engineer  existing  processes.  The 
new  system  does  not  differ  in  data  capture, 


front-end  editing  and  task  assignment/ 
oversight  when  compared  to  the  old  paper 
trail  and  manual  processes. 

The  next  phase  of  the  project  spiral  had 
many  options  for  demonstrating  how 
artificial  intelligence  (AI)  and  decision 
support  system  (DSS)  capabilities  can  be 
substituted  for  some  sub-processes  now 
done  by  people,  thereby  enabling  reductions 
in  support  infrastructure  and  improving 
overall  logistics  support.  For  example,  one 
option  was  to  demonstrate  how  AI  and  DSS 
could  improve  processing  by  using  a  front- 
end  data  editing  capability.  Front  loading 
with  an  expert  system  or  decision  support 
system  to  minimize  errors  in  data  entered 
into  the  system,  assign  the  careful 
distribution  of  data  among  the  systems,  and 
to  monitor  updates  to  the  databases  might 
achieve  significant  reductions  in  existing 
support  infrastructure. 

Another  option  considered  was  to  show  how 
a  decision  support  or  expert  system 
capability  might  enable  the  system  to 
intelligently  deal  with  any  future  changes  that 
may  occur  in  workflows  of  the  major 
support  processes,  i.e.,  (PMS,  ECPs,  APLs, 
SMDs,  TMs  and  training).  If  users  modify 
their  tasks,  changes  could  potentially  be 
propagated  to  the  database  or  the  business 
rules  of  the  organization  and  eventually 
support  AEGIS  re-engineering  efforts. 

The  third  option  (selected  for  demonstration) 
was  to  use  an  expert  system  to  meet 
requirements  for  the  storage  and 
implementation  of  the  customized  logistics 
information  processing  rules  of  the  Navy. 
These  are  the  rules  that  select  work 
processing  sequences  and  determine  how 
changes  to  logistics  documentation  are 
maintained.  A  complete  expert  system  might 
eventually  contain  hundreds  of  business  rules 
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for  updating  and  handling  the  enormous 
amount  of  logistics  documentation, 
engineering  drawings,  instructions,  and 
technical  materials  that  are  currently 
maintained  only  in  written  procedures  or  in 
the  corporate  knowledge  of  sailors.  Navy 
civilian  employees  or  contractor  support 
personnel.  Sometimes,  even  the  rules 
themselves  may  be  unwritten.  It  may  take 
months  or  years  to  train  a  sailor,  civilian 
government  employee  or  support  contractor 
to  become  completely  proficient  at  managing 
a  logistics  support  task  and  understanding 
the  rules.  Over  time,  the  system  could  be 
used  as  a  primary  logic  editor  for  the 
application  and  will  probably  increase  in  size 
as  the  ILS  systems  mature.  Data  to  be 
included  in  the  system  could  encompass 
training  information  and  related  data  that  are 
stored  in  job  descriptions  of  the  In-Service 
Engineering  Agent  (ISEA).  These 
documents  could  include  training  manuals, 
business  rules  for  each  system,  written 
guidelines  and  procedural  manuals. 

As  an  example,  the  system  will  seek  to 
demonstrate  how  it  can  be  used  as  the 
business  source  of  the  rules  that  define  what 
the  MCS  ISEA  does  to  sort  and  sequence 
maintenance  requests  for  system  technical 
documentation  and  manuals  when  ECP- 
driven  changes  are  processed.  The  expert 
system  will  demonstrate  how  it  can  provide 
data  to  the  workflow  management  system, 
which  checks  tables  to  determine  which  tasks 
are  to  be  performed  next.  If  a  status  flag 
shows  that  a  next  task  exists,  the  system  will 
place  the  logistics  support  data,  images, 
reports  and  associated  documents  in  the 
electronic  mailbox  of  the  cognizant 
individual  for  further  action. 

Using  this  capability,  the  Navy  could 
significantly  increase  managerial  ‘What  if’ 
capabilities  that  can  be  derived  from  the 


databases  and  environmental  information 
available  to  the  maintenance  personnel. 

FUTURE  METHODOLGY:  TAILORED 
TOOLS  AND  PROCEDURES  FOR 
INTEROPERABILITY 

Future  systems  development  methodologies 
must  utilize  tools  which  match  the  step-by- 
step  methodologies  to  the  needs  for  software 
and  systems  that  require  extensive 
integration  and  development  support.  The 
broad  functions  performed  by  the  tools  must 
cover  input  to  the  design  of  systems, 
diagramming  techniques,  design  specification 
which  can  produce  code  when  fed  into  code 
generators,  and  testing  and  debugging  tools 
to  speed  the  acceptance  and  testing  of 
software. 

Specialized  methods  will  be  required  to 
support: 

analysis  and  design  for  development 

and  integration 

database  and  file  design  for 

integration  data  modeling 

data  modeling  to  aid  the  maintenance 

of  integrated  systems 

programming  tools  for  code 

generation  and  testing 

project  management  for  planning  and 

controlling  complex  tasks 

The  goal  of  the  future  "detailed" 
development  methodologies  will  be  to 
provide  straightforward  and  easy-to-use 
instructions  and  technical  guidance  for  the 
implementation  of  systems  with  multiple 
components.  The  methodologies  will  help  to 
accomplish  this  goal  by  providing  clear 
developer  procedures  for  preparing  the  tools, 
detailing  step  by  step  instructions  indicating 
how  systems  can  be  integrated,  and  guidance 
for  the  integration  steps  necessary  to  link 
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diverse  systems  after  the  actual  development 
has  been  completed. 

SUMMARY 

The  evolving  DDG-51  information  systems 
environment  is  typical  of  organizations  that 
are  experiencing  an  ongoing  evolution  in 
their  information  systems,  concurrent  with 
changes  in  the  organization.  These 
information  systems  must  now  store 
information  in  an  organization  memory  that 
is  accessible  to  all  users  of  the  information  to 
serve  as  a  design  and  development  data 
dictionary;  and  to  support  the  ability  of 
organization  members  to  utilize  the  data 
model  as  a  tool  that  can  be  used  to 
characterize  the  business  relationships  among 
the  sub-systems  in  the  organization. 

This  means  that  the  system  must  be  capable 
of  identifying  the  impact  of  organizational 
changes,  identifying  overlaps  and  duplication 
in  data,  and  noting  where  and  how  task  re¬ 
engineering  might  impact  operations  by 
examining  the  business  relationships 
identified  through  the  data  stored  in  the 
systems  of  the  organization. 

The  question  of  "how"  to  accomplish  this  is 
a  primary  concern  for  managers  who  face 
unintegrated  systems  such  as  many  of  those 
in  today's  government  and  business 
environments.  Far  easier  is  to  describe  what 
has  happened  in  this  Navy  program  and  why 
the  change  appears  to  be  occurring  here, 
rather  than  to  proactively  state  how  new 
information  systems  can  be  developed  and 
effectively  implemented  for  similar  complex 
environments.  For  the  Navy,  the  change 
appears  to  have  been  both  strategic, 
incremental  and  evolutionary. 

The  implementation  of  the  product  locator 
and  its  workflow  components  is  being 


carefully  assessed  to  minimize  any  negative 
impact  on  the  DDG-51  project,  and  to 
implement  a  systems  strategy  to  transfer  this 
new  technology  throughout  the  AEGIS 
program.  Many  factors  seem  to  play  a  part 
in  making  the  success  occur.  The  timing  of 
the  change  was  a  clear  issue.  This  may 
include  the  business  and  technical  motivation 
of  managers  in  both  the  Information 
Resource  Management  (IRM)  organization 
and  in  the  AEGIS  Program’s  functional 
areas.  In  addition,  the  evolution  and 
availability  of  affordable  computer 
programming  tools,  including  the  support 
tools,  GUI  front  end,  database,  expert 
system,  business  rules,  optical  scanning  and 
storage  capability  all  appear  critically 
important.  For  the  AEGIS  Program,  price 
was  also  a  critical  issue.  The  high  cost  of 
mainframe  tools  would  have  made  the 
adoption  of  a  mainframe-based  CASE 
system  far  less  advantageous.  Managers 
should  continually  evaluate  the  relationships 
between  their  Information  Systems  (IS)  and 
business  environment,  and  develop  an 
ongoing  approach  to  strategic  planning  that 
involves  comprehensive  strategies  for 
adopting  CASE  and  other  tools  that  upgrade 
IS  business  and  organizational  support 
capabilities. 
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MISSION 

The  primary  mission  of  the  Defense 
Personnel  Support  Center  (DPSC)  is  to 
support  the  war  fighter  in  both  war  and 
peacetime  by  providing  quality  goods  at  fair 
and  reasonable  prices  in  the  most  timely 
fashion  available.  Everything  we  do,  from 
small  purchases  to  the  institution  of 
Electronic  Commerce  business  systems,  is 
done  to  support  this  critical  and  overarching 
responsibility.  Systems  and  communications 
are  simply  enablers  which  allow  us  to 
support  our  mission. 

VISION 

The  DPSC  vision  is  to  be  the  champion  of 
military  readiness,  worldwide  provider  of 
federal  logistical  services  and  a  leader  in 
business  innovation.  We  have  combined  the 
essential  objectives  of  our  mission  -  combat 
readiness  and  customer  response  -  and 
placed  the  highest  priority  on  their 
attainment,  which  encompasses  having  the 
right  items,  at  the  right  time,  place  and  cost. 
To  attain  these  objectives,  our  quantum  leap 
strategy  is  the  DPSC  Logistics  R&D 
Program  called  (CR)2;  it  is  our  blueprint  for 
the  future. 

INTRODUCTION 

For  many  years,  the  national  defense 
logistics  system  depended  on  massive 
inventories  of  unique  items  produced  to 
stringent  military  specifications.  We 
supported  our  customers'  readiness  and 
peacetime  requirements  by  using  DoD's 


leveraged  buying  power  to  receive 
substantial  discounts  on  stock  that  we  then 
held  in  government  warehouses  until 
customers  requisitioned  it.  By  taking 
advantage  of  economies  of  scale,  we  were 
able  to  provide  our  customers  with  product 
at  prices  substantially  below  those  they 
could  acquire  for  themselves.  In  fiscal  year 
1994,  our  total  sales  were  over  $3.4  billion, 
an  amount  that  placed  us  well  ahead  of  many 
of  the  Fortune  500  companies. 

Historically,  we  conducted  much  of  our 
business  on  paper.  Our  business  strategy 
was  based  on  the  assumption  that  holding 
inventory  in  our  warehouses  provided  our 
customers  with  acceptable  service  and  the 
lowest  delivered  product/service  costs.  We 
also  assumed  that  awarding  relatively  short¬ 
term  contracts  to  vendors  on  the  basis  of 
lowest  price  would  achieve  the  lowest 
delivered  cost.  In  reality,  that  process  was 
slow,  prone  to  error,  and  expensive. 
Customers  routinely  received  product  60 
days  after  ordering  with  a  considerable 
surcharge  (up  to  28%)  added  to  the  price  of 
the  item. 

READINESS  IN  THE  NEW  WORLD 
ORDER 

The  greatest  challenge  facing  the  DPSC 
and  in  fact,  the  entire  DoD,  is  how  to 
maintain  a  readiness  posture  in  an  era  of 
reduced  budgets,  reduced  peacetime  needs 
and  increased  competition.  In  order  to 
support  wartime  requirements,  the  DPSC 
must  sustain,  through  contracts  for 
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peacetime  requirements,  a  robust  logistics 
support  system  to  include  raw  material 
suppliers,  manufacturers  and  distributors 
capable  of  responding  to  rapid  changes  in 
demand.  The  DPSC  is  facing  an  increasing 
threat  to  its  reduced  peacetime  sales  base  due 
to  increased  competition  from  other  Federal 
agencies  and  the  private  sector.  The  erosion 
of  our  sales  base  results  in  the  erosion  of  our 
ability  to  support  an  industrial  base 
sufficient  to  meet  our  national  defense  and 
emergency  readiness  requirements. 

The  current  business  environment 
demands  quicker  response  to  marketplace 
changes,  greater  efficiencies  and  lower  costs 
of  doing  business.  Adjusting  to  that  reality 
led  us  to  fundamentally  change  the  way  we 
conduct  our  business.  We  realized  that  our 
processes  and  the  assumptions  on  which  we 
operated  were  not  always  valid.  Like 
industry,  we  now  focus  on  long-term 
relationships  with  our  business  partners  to 
give  customers  what  they  want,  when  they 
want  it,  and  at  a  price  they  are  willing  to  pay. 

The  DPSC  achieved  a  higher  level  of 
customer  service  through  the  use  of 
Electronic  Commerce  (EC)  initiatives 
coupled  with  reengineered  business  and 
logistics  practices.  These  strategies  offer  an 
opportunity  for  the  DPSC  to  not  only 
become  the  supplier  of  choice  to  the 
military,  but  to  also  expand  our  sales  base  to 
include  other  non-traditional  customers,  thus 
enabling  us  to  sustain  ourselves  during 
peacetime  to  be  ready  for  our  national 
defense  and  emergency  responsibility. 


BUSINESS  ENVIRONMENT 

REENGINEERING 

We  define  reengineering  as  a 
fundamental  rethinking  and  radical  redesign 
of  business  practices  resulting  in  dramatic 
improvements  in  customer  satisfaction.  In 
our  reengineering  efforts,  we  attempt  to  link 
all  re-design  efforts  to  corporate  objectives, 
take  a  win-win  approach  with  all  our  trading 
partners,  reengineer  processes  before 
systems,  and  always  remain  customer 
driven.  The  ultimate  goal  is  to  do  business 
in  our  industries  in  the  way  they  do  business. 
To  achieve  this  goal,  we  emulate  the  best 
commercial  practices  without  imposing 
government  unique  requirements. 

ELECTRONIC  COMMERCE 

Of  all  the  changes  we  are  making  to  attain 
excellence  in  logistics  support,  integrating 
the  use  of  Electronic  Data  Interchange  (EDI) 
and  other  enabling  technologies  into  our 
business  systems  to  achieve  total  electronic 
end-to-end  business  transactions  or 
Electronic  Commerce,  is  the  most 
significant.  It  has  allowed  us  to 
fundamentally  change  our  business  and 
logistics  processes  and  approaches  in  all  of 
our  functional  areas. 

Electronic  Commerce  is  an  advanced 
total  business  system  that  provides  a  solution 
to  implementing  our  corporate  business 
strategies.  These  strategies  move  beyond 
conventional  practice  and  experience.  The 
system  takes  full  advantage  of  the 
opportunities  available  through  commercial 
acquisition  methods  utilizing  EDI 
technologies.  We  have  taken  and  tailored 
the  best  ideas  from  several  industries  to 
develop  our  strategy. 

The  DPSC  Electronic  Commerce 
Program  was  designed  to  emulate  successful 
commercial  industry  electronic  commerce 
models.  Electronic  Commerce,  sometimes 
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referred  to  as  electronic  trading,  is  a  total 
business  system  which  maximizes  the  use  of 
information  to  hasten  the  movement  of 
money  and  merchandise  among  long-term 
trading  partners.  One  of  the  tenets  of  EC  is 
the  use  of  EDI,  which  is  an  enabling 
technology  that  permits  the  flow  of 
information  in  a  paperless  environment 
through  the  use  of  standard  formats  evolved 
through  industry  consensus.  These  model 
systems  use  long-term  trading  partnerships 
integrating  suppliers,  vendors, 

manufacturers,  distributors  and  end  use 
customers  in  an  EDI  network. 

Electronic  trading  requires  a  business 
system  that  permits  the  public  sector  to  do 
business  in  the  way  each  of  its  industries 
does  business,  i.e.,  using  standard  industry 
practices  and  EDI  formats  and  deemphazing 
the  uniqueness  of  government  requirements 
and  specifications.  It  is  for  this  reason  that 
we  utilize  the  industry  convention  rather 
than  a  special  government  convention.  At 
DPSC,  we  have  the  capability  to 
communicate  with  trading  partners  in  a 
variety  of  ways,  e.g.,  using  Value  Added 
Networks  (VANs),  direct  connections  and 
government  hubs.  Our  success  is  based  in 
flexibility  and  adopting  commercial, 
industry  developed  systems  and  standards. 

DPSC  is  at  the  forefront  in  the 
development  and  demonstration  of 
Electronic  Commerce  business  applications. 
Our  pioneering  efforts  in  the  public  sector 
application  of  EDI  are  recognized  by  the 
Price  Waterhouse  Corporation.  In  its 
landmark  benchmarking  study  of  EDI 
business  systems  applications,  DPSC  is  the 
only  government  invitee  in  a  select  group  of 
24  nationally  recognized  leaders  in  EDI  such 
as  JC  Penney,  IBM,  Motorola  and  Texaco. 
Now  in  its  second  year  of  benchmarking, 
DPSC  continues  to  place  about  in  the  middle 
of  this  prestigious  group. 


The  application  of  EDI  technologies  to 
the  large  purchase  arena  (over  $25,000), 
which  constitute  the  bulk  of  DPSC 
purchases,  is  where  true  EC  operates.  The 
concept  of  electronic  trading  moves  beyond 
the  simple  movement  of  information  to 
include  merchandise  and  money  and  deals 
with  a  variety  of  business  concepts  that 
affect  a  number  of  business  functions 
including  transportation,  inventory 
management,  finance,  and  purchasing. 
These  relationships  are  complex  in  nature 
requiring  suppliers  to  not  only  possess  the 
capability  to  conduct  EDI,  but  to  also 
establish  innovative,  robust  business  systems 
that  use  EDI  as  a  tool.  Examples  are  our 
Prime  Vendor  and  Quick  Response 
Programs,  where  due  to  the  rapid  flow  of 
information,  suppliers  can  guarantee  specific 
delivery  requirements  (some  routinely  in 
less  than  24  hours)  and  adjust  production 
based  on  actual  usage.  Here  EDI,  the  tool, 
has  permitted  changes  in  inventory 
management,  distribution  and  financial 
processes  to  move  merchandise  and  money 
as  well  as  data. 

BUSINESS  PRACTICES 

Gone  are  the  days  of  700  page 
specifications  for  chocolate  chip  cookies  and 
overloaded  warehouses  of  1950's  vintage 
hospital  gowns  purchased  based  on 
erroneous  forecasts.  Today  the  DPSC  buys 
commercial  products  manufactured  in 
response  to  actual  demand,  delivered  direct 
to  the  customer's  door  in  a  matter  of  days 
instead  of  the  months  previously  required  for 
delivery  alone.  Our  objective  is  to  either 
buy  commercially  available  items  or 
manufacture  to  actual  demand  as  it  is 
recorded  at  the  point  of  sale  or  issue. 

Three  specific  examples  of  reengineered 
logistics  processes  are  "Quick  Response", 
"Shared  Production"  and  "Prime  Vendor". 
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QUICK  RESPONSE 

Quick  Response  is  the  effective  use  of 
Electronic  Data  Interchange  (EDI)  coupled 
with  modern  inventory  management  and 
manufacturing  techniques  to  speed  the  flow 
of  information  and  supplies  among  the 
business  partners.  Point  of  sale  data  is 
collected  from  our  customer  and  sent  via 
EDI  to  the  manufacturer's  integrated 
automated  information  system.  The  data  is 
used  for  production  planning.  Computer 
aided  design  and  manufacturing  technology 
is  used  to  transmit  technical  documents  and 
patterns.  Flexible  or  modular  manufacturing 
systems  are  employed  to  permit 
manufacturers  to  adjust  production 
schedules  as  needed  to  ensure  adequate  stock 
replenishment.  Delivery  takes  place  within 
72  hours  after  order.  Supplies  are  shipped 
directly  to  the  customer  instead  of  to  a 
government  warehouse.  Since 

implementing  Quick  Response  customers 
have  saved  over  $1  million  in  reduced 
government  inventory  and  handling  costs. 

The  flexibility  provided  by  the  Quick 
Response  system  permit  manufacturers  to 
quickly  surge  production  in  response  to 
emergencies.  The  system  was  tested  in 
October  1994  when  Iraq  once  again  posed  a 
threat  to  Kuwait.  Three  Quick  Response 
vendors  began  manufacturing  and  shipping 
tens  of  thousands  of  fatigue  uniforms  within 
ten  days  after  order.  Within  four  weeks, 
100,000  sets  of  fatigues  were  prepared  and 
ready  for  any  troops  that  might  be  deployed. 
The  greatest  endorsement  of  the  Quick 
Response  system  came  from  a  satisfied 
customer  who,  after  receiving  a  desperately 
needed  item  in  five  hours  after  order, 
exclaimed  that  it  was  a  feat  "nothing  short  of 
a  miracle". 


SHARED  PRODUCTION 

What  do  the  McDonald's  Corporation, 
NIKE,  Nestle,  Hard  Rock  Cafe  and  the 
Department  of  Defense  have  in  common? 
Long-term  partnerships  to  ensure  defense 
readiness  in  a  changing  world  order.  Lauded 
by  Connie  Chung  of  CBS  News  as  "an 
example  of  defense  conversion  at  its  best", 
the  Shared  Production  Agreement  concept 
represents  a  major  conceptual  change  in  the 
defense  planning  process.  The  Shared 
Production  Agreement  concept,  pioneered 
by  the  DPSC,  partners  manufacturers  with 
the  DoD  and  other  customers  to  ensure 
production  capacity  for  military 
requirements  during  national  emergencies. 

Under  the  Shared  Production  Agreement 
concept,  commercial  capacity  and 
inventories  are  shared  by  the  DoD  and  major 
non-defense  customers  such  as  McDonald's 
and  NIKE,  etc.,  who  agree  to  share  the 
production  facility  during  peacetime  or 
times  of  normalized  requirements  and 
facilitate  dedicated  production  during  surges 
in  demand  by  either  sharing  partner.  In  the 
event  of  a  national  emergency,  the 
commercial  customer  agrees  to  relinquish 
production  lines  and  capability,  as  necessary, 
and  return  to  its  normal  business  relationship 
with  the  vendor  as  military  requirements 
decrease.  By  sharing  the  production  facility, 
the  manufacturer's  dependence  on 
government  contracts  is  reduced  and  defense 
surge  capability  is  maintained  and/or 
expanded  at  a  reduced  cost.  This  approach 
also  ensures  that  contracts/orders  in  support 
of  the  national  defense  will  be  accepted  and 
given  priority,  and  makes  industrial  base 
planning  with  the  DoD  more  attractive  to 
commercial  companies. 

Rapid  conversion  requires  the  items 
produced  for  both  the  military  and 
commercial  customer  be  produced  using 
similar,  generally  accepted  commercial 
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manufacturing  practices,  by  a  workforce 
cross-trained  and  tested  in  both  military  as 
well  as  other  customer  production.  Agile  or 
modular  manufacturing  and  EDI  are  the 
enabling  technologies.  These  methodologies 
allow  manufacturers  to  produce  both  large 
and  small  production  runs  of  complex  and 
critically  needed  items  and  deliver  them 
directly  to  designated  customers  in  an 
Electronic  Commerce  system. 

The  Shared  Production  program  is 
supported  by  seven  agreements  to  date  and  is 
growing.  Replication  of  the  Shared 
Production  concept  is  being  encouraged 
throughout  DoD. 

PRIME  VENDOR 

The  Prime  Vendor  Program  is  a 
commercial  distribution  system  that  allows 
our  customers  to  exercise  their  discretionary 
buying  power  to  order  only  what  they  need, 
when  they  need  it.  Customers  choose  the 
medical  or  food  item  of  their  choice  from  an 
electronic  catalog,  and  transmit  the  order 
electronically  direct  to  the  "Prime  Vendor". 
Confirmation  of  the  order  is  received  in 
minutes  and  delivery  occurs  within  24  hours. 
Payment  is  accomplished  through  electronic 
invoicing  and  Electronic  Funds  Transfer 
(EFT)  making  Prime  Vendor  a  total  end-to- 
end  Electronic  Commerce  business  system. 

The  DP  SC  negotiates  the  contracts  that 
underpin  the  program,  performs  contract 
administration  and  pays  the  bill  for  the 
customer  all  for  a  minimal  1%  service  fee. 
Leveraged  buying  allows  the  customer  to 
take  advantage  of  lower  prices  without 
investing  capital  in  massive  inventories.  The 
program  shifts  the  onus  for  holding  and 
managing  inventory  to  the 
manufacturer/distributor.  The  only 

inventory  that  will  be  held  in  government 
warehouses  are  those  quantities  necessary  to 
support  emergencies  until  surge 
agreements/options  are  activated. 


The  use  of  commercial  items  and 
distribution/inventory  systems  provides  a 
higher  level  of  flexibility  and  customer 
response.  A  medical  Prime  Vendor 
delivered  critical  medical  supplies  to  the 
scene  of  a  tragic  airplane  crash  at  Pope  Air 
Force  Base  in  North  Carolina  within  30 
minutes  of  the  incident;  an  impossible  feat 
under  previous  distribution  methods. 

The  Prime  Vendor  concept  was  initially 
piloted  in  the  medical  pharmaceutical  arena. 
The  program  has  since  been  rolled  out  to 
include  medical  surgical,  food  service  and 
clothing  items.  Within  DPSC,  Prime 
Vendor  currently  accounts  for  60%  of 
medical  and  20%  of  its  food  service  sales 
and  is  growing.  The  high  level  of  customer 
choice  and  service  provided  by  the  Prime 
Vendor  Program  has  enabled  the  DPSC  to 
retain  sales  threatened  by  the  highly 
competitive  environment  in  addition  to 
capturing  new  sales. 

CHALLENGE 

Our  national  defense  readiness  strategies 
now  heavily  rely  on  immediate  industry 
surge  response  and  shared  production 
agreements  made  possible  through  our 
electronic  commerce  business  systems.  It  is 
crucial  to  our  readiness  posture  that  we 
partner  with  suppliers  who  will  enter  into 
long-term  trading  partner  relationships  with 
us  to  ensure  product  availability  during 
national  emergencies.  The  loss  of  even  one 
critical  supplier  may  have  serious  national 
defense  readiness  implications.  The 
remaining  challenge  is  to  fully  integrate 
these  successfully  demonstrated  new 
logistics  approaches  into  our  readiness 
planning  models  and  total  business  system. 
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FUTURE 

ACHIEVING  READINESS  UTILIZING 
COMMERCIAL  LOGISTICS  SYSTEMS 
(CR)2 

The  Defense  Personnel  Support  Center  is 
a  leader  in  both  Government  and  the  private 
sector  in  the  use  and  development  of 
Electronic  Commerce.  The  management  at 
DPSC  has  successfully  re-engineered 
business  practices  and  processes  in  all 
commodity  business  units.  Enabling 
technologies  such  as  EDI  have  been 
implemented,  paving  the  way  for  a  totally 
electronic  commerce  environment  in  the 
future.  The  enhancements  we  have 
instituted  through  "Buy  Response  Vice 
Inventory"  and  other  incremental  process 
improvements  have  refined  the  way  we  do 
business  today.  However,  like  all  successful 
businesses,  we  are  planning  today  for  the 
long  range  strategy  that  will  carry  us  through 
the  21st  Century  and  allow  us  to  surpass  the 
competition. 

We  have  embarked  on  aggressive,  bold 
new  business  practices  which  when  totally 
deployed  will  change  our  business  processes. 
Previous  processes  which  segregated 
functions  such  as  requisition  processing, 
cataloging,  contracting,  distribution  and 
customer  service  will  be  performed 
interactively  and  information  related  to  these 
processes  will  be  stored  and  available  in  an 
interactive  mode.  Therefore,  it  is  necessary 
to  develop  a  repository  for  all  cataloging, 
technical,  quality  and  contracting 
information  which  can  be  viewed  and 
utilized  for  a  variety  of  purposes.  Rather 
than  partition  data  into  functional  units,  an 
on-line  real  time  interactive  database  must 
be  developed  which  not  only  services  the 
purposes  of  DPSC  for  management  of  data, 
but  also  serves  as  a  contracting  tool  for 
customers  and  a  repository  of  public 
information  for  industry.  An  interface  with 


an  automated  ordering  system  will  be  the 
path  to  the  contracting  tool.  The  system 
would  act  as  a  logistics  manager,  allowing 
DPSC  to  manage  items  and  distribution 
methods  while  customers  would  actually  be 
able  to  place  orders  using  the  system. 
Systems  integration  and  enhancement  is 
required  to  allow  DPSC  to  maintain  its 
position  as  a  leader  in  customer  support  and 
combat  readiness. 

GOALS 

The  (CR)^  initiative  seeks  to  accomplish 
three  goals: 

*  posture  DPSC  for  a  higher  level  of 
combat  readiness  in  the  new  business 
environment; 

*  make  DPSC  an  artery  for  customer 
and  supplier  entry/exit  on  the  information 
superhighway; 

*  explore  state  of  the  art  and  leading 
edge  advanced  information  technology  for 
potential  application  and  benefit  throughout 
DoD  and  the  Defense  Logistics  Agency. 

COMBAT  READINESS 

The  combat  readiness  portion  of  (CR)^ 
will  empower  logistics  planners  to  forecast  a 
sustainment  scenario  in  accordance  with  a 
specific  force  package.  Modeling  and 
hypothetical  situation  analysis  will  be  used 
extensively  to  determine  optimum  solutions 
to  current  and  future  readiness  dilemmas. 

The  plans  to  support  two  nearly 
simultaneous  Major  Regional  Conflicts 
(MRC)  specified  by  the  current  National 
Military  Strategy  will  be  addressed  in  (CR)^. 
We  will  make  assumptions  about  wartime 
requirements  consistent  with  past  experience 
and  current  projections.  Additionally,  we 
will  use  actual  demand  history  from  the  most 
recent  conflicts,  humanitarian  relief  efforts, 
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and  the  two-MRC  scenario  to  identify  a 
second  echelon  of  potential  warstopper 
items.  The  (CR)^  algorithm  will  offset  these 
requirements  with  projected  assets  (on  hand 
and  due  in)  from  depots  and  the  Services. 
The  algorithm  will  further  reduce  the 
requirement  by  the  amount  that  the  industrial 
base  is  capable  of  providing.  The  residual 
requirements  represent  the  inventory 
investment  or  business  arrangements  that 
must  be  supported  to  bridge  the  gap  between 
D-Day  and  the  day  that  production  can  meet 
consumption. 

(CR)2  will  be  developed  and  tested  in 
labs  which  will  enable  DPSC  to  share 
information  and  partner  with  other  agencies. 
For  example,  we  will  partner  with  Advanced 
Research  Projects  Agency  (ARP  A)  to 
incorporate  specific  technologies  that 
enhance  and  empower  our  processes. 

CUSTOMER  RESPONSE 

The  DPSC  Customer  Response  -  Combat 
Readiness  (CR)2  System  will  reside  at  an 
exit  on  the  Information  Superhighway  and 
will  be  accessible  to  any  and  all  interested 
parties,  to  include  military  customer, 
manufacturers,  dealers,  industry 
associations,  as  well  as  public  and  private 
institutions.  Access  to  the  system  would  be 
the  same  as  entry  onto  the  Information 
Superhighway;  through  state  of  the  art 
digital  technology  whether  that  be  telephone 
modem  or  electronic  network 
communications.  Information  will  be 
partitioned  into  views  to  enable  preferred 
customers  unlimited  access  to  data.  Casual 
shoppers  and  competitors  will  also  be 
afforded  a  view,  but  the  scope  and  breadth  of 
data  will  be  a  mere  glimpse  of  what  actually 
resides  at  that  exit. 

Preferred  customers  will  be  able  to 
browse  or  purchase,  gathering  information  at 
various  levels  of  desired  detail.  Immediately 
upon  access  to  our  system,  shoppers  will  be 


identified  by  their  customer  number.  Items 
will  be  easily  called  up  by  a  variety  of 
lexicons,  and  searches  will  be  tailored  to  the 
level  of  detail  desired.  Customers  will  see 
similar  and  identical  products  offered  via 
differing  methods  of  distribution,  and  will  be 
able  to  make  best  value  source  selection 
decisions  based  on  their  individual  needs. 
Prices  will  be  displayed  for  various 
distribution  methods  and  delivery  times, 
enabling  the  customer  to  decide  on  the  best 
selection.  If  desired,  the  shopper  will  be 
able  to  actually  "see"  the  product  in  three 
dimensional  presentations.  The  system  will 
incorporate  Computer  Aided  Design 
software  to  enable  users  to  draw  and  display 
products. 

Additionally,  technical  information 
concerning  standards  utilized  in  product 
manufacturing  processes,  dimensions,  and 
other  product  information  will  be  displayed. 
Information  on  quality  history  will  also  be 
available  as  well  as  recall  and  regulatory 
agency  updates. 

Customers  and  logisticians  will  have 
access  to  virtual  inventories  by  viewing  on 
hand  stock  levels  at  government  and 
commercial  locations,  i.e.,  storage  locations 
and  distribution  sites.  This  visibility  will 
enable  customers  to  make  decisions  on  when 
to  buy  and  how  much  to  buy.  Also,  (CR)2 
removes  the  variance  from  the  business 
environment.  Buyers  of  goods  will  know 
specifically  when  product  will  be  delivered 
to  them;  they  will  have  a  choice  of  delivery 
modes  and  related  costs,  however,  there  will 
be  certainty  associated  with  the  choice.  To 
explain,  if  the  buyer  chooses  to  wait  three 
weeks  to  receive  goods  from  a  low  cost 
supplier,  that  is  the  buyer's  prerogative. 
However,  he  will  know  WITH 
CERTAINTY  that  the  product  will  be 
delivered  on  a  specific  day.  That  certainty 
does  not  exist  in  today's  environment. 

Data  entrance  and  exit  to  the  system  will 
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only  be  possible  through  the  exclusive  use  of 
EDI  standard  transaction  data  sets.  Owners 
of  data  will  be  responsible  for  ensuring 
accuracy  while  systems  edits  will  protect  the 
database  from  incomplete  data  sets  entering 
the  highway.  Data  calls  will  enable 
regulatory  agencies  to  update  new  item 
entries  with  quality  history  prior  to  the  item 
being  available  for  wholesale  viewing. 

The  system  will  run  in  a  real  time  on-line 
environment.  Commercial  product 

classification  software  will  be  utilized  to 
group  like  items  to  help  customers  shop  in 
an  orderly  fashion  without  having  to  know 
the  item  number  or  stock  number.  When 
customers  place  an  order,  demand  data  will 
update  the  Customer  Demand  Management 
Information  system.  Also,  depending  on  the 
distribution  method  selected,  inventory 
adjustments  will  be  made  and  orders  for 
replenishment  will  be  triggered  either  to 
DPSC,  Central  Distribution  Centers  (CDCs) 
or  even  Prime  Vendors,  to  name  a  few  of  the 
choices. 

Customers  will  enter  the  highway,  exit  at 
our  database,  browse  and  shop,  select  an 
item  and  method  of  delivery,  enter  a  quantity 
and  hit  the  order  key.  Immediate 
confirmation  of  the  order  with  a  delivery 
date  and  an  order  number  will  occur.  The 
customer  will  be  asked  if  the  shipping 
address  on  file  is  the  address  desired.  If  it  is 
not,  an  order  specific  address  can  be  entered. 
He  then  drives  off. 

(CR)2  -  THE  SYSTEM 

(CR)2  will  result  in  quantum 
improvements  in  combat  readiness  and 
customer  response  by  integrating  operations 
across  present  commodity  boundaries.  The 
program  will  encompass  all  existing  systems 
and  processes,  resulting  in  a  seamless 
entry/exit  into  DPSC  information  and 
technology.  The  primary  focus  of  (CR)2  is 
to  streamline  and  automate  the  interface 


between  the  private  sector  and  the  item's  end 
user,  the  war  fighter. 

(CR)2  will  integrate  emerging  R&D 
concepts  into  an  overall  revolutionary 
logistics  system.  One  of  the  primary  issues 
is  one  of  scope.  Many  emerging 
technologies  that  work  well  on  a  small  scale 
and  limited  scope  break  down  when  the 
scale  and  scope  are  significantly  expanded. 
The  (CR)2  initiative  expands  the  use  of 
emerging  research  to  a  larger  scale  and 
scope  and  tests  them  in  a  scaleable 
operational  environment. 

(CR)2  will  integrate  our  processes  and 
systems  by  exploiting  the  natural  synergism 
among  our  Commodity  Business  Units.  This 
will  be  accomplished  through  cross 
pollination  of  proven  technologies  that  have 
reached  various  stages  of  maturity  within 
narrow  operational  areas.  These 

technologies  will  be  expanded,  modified  and 
enhanced  to  embrace  a  larger  group  of  users. 

PARTNERSHIPS 

(CR)2  will  be  developed  and  tested  in 
labs  which  will  enable  DPSC  to  share 
information  and  partner  with  other  agencies 
to  incorporate  specific  technologies  that 
enhance  and  empower  our  processes. 

DPSC  will  invite  the  Services,  DLA,  our 
trading  partners,  and  the  information 
technology  industry  to  participate  in  (CR)2. 
Consistent  with  DPSC's  current  status  as  a 
DoD  Demonstration  Center  for  acquisition 
reform,  best  commercial  practices,  and 
electronic  commerce,  DPSC  welcomes  the 
participation  of  other  DoD  components  as 
well  as  other  Federal  agencies. 

SAVINGS 

The  Defense  Personnel  Support  Center  is 
part  of  Defense  Logistics  Agency,  which  is  a 
combat  support  agency  for  consumable 
items.  Each  Military  Service  will  benefit 
directly  from  improvement  in  DLA's  ability 
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to  support  the  war  fighter  with  shorter  lead- 
times  and  reduced  cost.  (CR)^  allows  DPSC 
to  develop  and  test  large  scale  projects 
related  to  the  logistics  mission.  Based  on  the 
results  of  these  tests,  gaps  in  current  research 
and  advanced  technologies  that  are  mature 
enough  for  operational  implementation  are 
identified.  The  proposed  development  is  a 
high  risk,  high  payoff  alternative  to  the 
logistics  systems  being  developed  and 
implemented  through  Corporate  Information 
Management  and  DBOF  funding.  The 
program  takes  a  holistic  approach  to  the 
logistics  system  including  the  heavy  reliance 
that  will  be  placed  on  the  private  sector  for 
privately  held  inventory.  The  most 
significant  outcomes  include  improved 
combat  readiness  and  customer  choice  in 
terms  of  items,  source,  cost/schedule 
tradeoff,  and  reduced  transaction  costs,  as 
well  as  quantrum  improvements  in  Logistics 
Response  Time  (LRT). 

Improved  Response:  This  will  be 

accomplished  through  the  synergy  of  the 
electronic  catalog  and  Electronic  Data 
Interchange  technology.  Customers  will 
access  the  electronic  catalog  directly, 
avoiding  the  delays  associated  with 
requisition  processing,  and  then  experience 
instantaneous  order  entry  through  the 
technology  of  EDI.  As  soon  as  an  order  is 
placed  by  a  customer,  an  850  transaction  is 
sent  to  the  manufacturer/distributor. 
Customers  receive  immediate  status 
regarding  delivery  date  and  mode  of 
transportation. 

Improved  Qistomer  Support:  Improved 
distribution  methods  such  as  Prime  Vendor, 
Dedicated  Truck,  Cross  Docking  and 
FEDEX  Air  will  be  employed  as  part  of  the 
business  arrangements  supporting  the 
various  price/quality  choices.  This  will 
result  in  improved  delivery  time  to 


customers. 

Lower  Prices :  Negotiation  of  long  term 
contracts,  standing  offers,  distribution  and 
pricing  agreements,  supply  bulletins  and 
other  innovative  acquisition  instruments 
result  in  better  deals  for  our  customers. 
Industry  is  receptive  to  new  approaches  to 
contracting  and  has  committed  to  offering 
best  customer  prices.  DPSC,  acting  as  the 
group  purchaser  for  its  customers,  receives 
deep  discounts  due  to  volume,  use  of 
electronic  order  entry  systems,  and  use  of 
electronic  funds  transfer. 

Reduced  Inventories  (at  both  retail  and 
wholesale  levels):  As  DPSC  moves  to  a 
direct  vendor  delivery  environment,  drastic 
inventory  reductions  will  occur  at  both  the 
retail  and  wholesale  levels. 

Reduced  Warehouse  Space;.  As  DPSC 
embraces  direct  vendor  delivery 
methodologies  and  just  in  time  inventory 
management  techniques,  there  will  be  little 
need  for  traditional  warehousing  of 
government  owned  inventory.  This  results 
in  decreased  overhead  expenses,  allowing  us 
to  lower  the  surcharge  to  our  customers 
while  improving  support  in  terms  of  faster 
delivery. 

Less  Infrastructure :  As  DPSC  adopts 

commercial  buying  practices,  the  need  for 
catalogers,  technical  writers,  engineers,  item 
manager,  quality  assurance  specialists,  and 
other  functional  specialists  decreases  while 
the  Center  becomes  a  contracting  activity. 
Generalists  will  be  employed  to  manage  a 
multitude  of  functions  associated  with  the 
logistical  support  of  customers  rather  than 
diverse  groups  of  functional  specialists 
micro-managing  parts  of  the  process. 
Responsibility  and  authority  will  rest  in  the 
multi-functional  team  rather  than  in  separate 
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parts  of  the  organization. 

MIS  System.  Development  of  an  integrated 
system  will  allow  DPSC  to  develop  a 
Management  Information  System  for  all 
CBUs;  this  does  not  exist  in  today's 
fragmented  environment. 

Recovery:  Use  of  a  single  system  to  enter 
and  view  data  will  mean  that  significantly 
fewer  people  are  required  for  recovery  if  the 
physical  data  processing  platform  were  to  be 
destroyed.  In  today's  environment, 
individuals  who  know  existing  legacy 
systems  are  needed  for  recovery.  With 
(CR)2,  systems  are  integrated  and  entry  and 
exit  is  uniform  for  all. 

METRICS 

In  order  to  measure  success,  metrics  are 
identified  at  the  outset  and  can  be  tracked 
during  the  evolution  of  the  program.  For 
(CR)2,  the  metrics  we  will  track  are: 

Combat  Readiness  -  we  will  track 
improvements  in  our  readiness  posture  in 
terms  of  "percent  ready".  This  metric  will 
look  to  the  establishment  of  business 
arrangements  required  to  support  two 
MRCs.  The  critical  items  will  be 
established,  quantities  required  will  be 
calculated  and  then  virtual  inventories  will 
be  measured.  The  delta  between  existing 
private  stocks  and  the  value  required  to 
support  two  MRCs  is  the  critical  value  to  be 
measured.  This  must  be  covered  through 
inventories  (Gov't,  virtual),  shared 
production  arrangements,  stock  rotation 
contracts,  surge  clauses  and  other  business 
arrangements  which  when  invoked,  will 
support  readiness  at  100%. 

Customer  Response  -  we  will  track 
improvements  in  the  consistency  of  support 
by  measuring  the  variance  experienced  by 
customers.  (CR)2  will  remove  the  variance 


in  delivery  time  by  forecasting  at  100% 
accuracy  when  customers  will  receive  goods. 

CONCLUSION 

In  the  current  climate  of  downsizing  in 
both  manpower  and  inventory  investment 
funds,  it  is  crucial  that  we  develop  new 
strategies  for  maintaining  our  readiness 
posture.  (CR)2  will  provide  the  solutions  to 
current  readiness  issues  and  posture  us  to 
quickly  respond  in  an  ever  changing  global 
environment. 
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ABSTRACT 

FedEx  and  other  private-sector  overnight 
package  delivery  services  could  not  achieve 
their  level  of  service  and  profitability  without 
using  bar  codes.  Government  managers  and 
managers  in  private-sector  companies  can  use 
the  same  technology  to  understand  how  work 
flows  within  their  organizations.  This  technol¬ 
ogy  aids  organizations  in  developing  process 
flow  information  and  in  business  reengineering 
and  streamlining  administrative  processes.  We 
applied  bar  code  technology  to  the  procurement 
process  of  a  medium  sized  nonprofit  organiza¬ 
tion  to  test  our  premise  that  government  and 
commercial  operations  can  also  benefit  from 
that  technology. 

We  reached  the  following  conclusions  re¬ 
garding  bar  codes 

•  They  are  an  effective  tool  for  identify¬ 
ing  and  validating  process  flow  steps. 

•  They  provide  the  means  for  easily  de¬ 
veloping  the  database  needed  to  gen¬ 
erate  a  variety  of  on-line  management 
performance  reports. 

•  They  save  time  and  improve  the  accu¬ 
racy  of  data  entry. 

•  They  are  an  effective  device  for  track¬ 
ing  and  locating  documents. 

We  believe  bar  code  technology  can  be 
equally  effective  when  applied  to  other  adminis¬ 
trative  processes. 


INTRODUCTION 

FedEx  (formerly  Federal  Express)  and 
other  overnight  package  delivery  services  could 
not  achieve  their  level  of  service  and  profitabil¬ 
ity  without  using  bar  codes.  That  technology 
has  permitted  them  to  maintain  complete  con¬ 
trol  of  millions  of  items  at  all  times  even  though 
many  items  travel  thousands  of  miles  each. 
Using  the  data  generated  from  bar  code  readers, 
FedEx  is  able  to  track  the  locations  of  any  of 
their  packages  at  any  time.  Government  man¬ 
agers  and  managers  in  private-sector  companies 
can  use  the  same  technology  to  understand  how 
work  flows  within  their  organizations.  Addi¬ 
tionally  they  can  use  bar  codes  to  create  a  data 
base  to  support  management  reports  and  docu¬ 
ment  control. 

This  paper  describes  the  use  of  bar  codes 
in  the  procurement  process  to  increase  manage¬ 
rial  effectiveness  and  efficiency  by  providing 
better  tools  for  controlling  the  flow  of  work. 
That  control  should  also  ensure  optimal  use  of 
scarce  resources.  At  the  Logistics  Management 
Institute  (LMI),  we  recently  completed  the  de¬ 
velopment  and  testing  of  a  system  using  bar 
codes  to  track  activities  in  the  procurement 
process  as  a  prototype  for  Federal  agencies. 
That  system  serves  as  a  test  bed  to  determine 
the  usefulness  of  bar  codes  for  improving  the 
ability  of  managers  and  their  staffs  to  identify 
the  work  process  flow  and  to  streamline  and 
simplify  it,  to  improve  the  usefulness  of  the  re¬ 
porting  system,  and  to  continually  track  all 
documents. 
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In  the  past  few  years,  many  papers  and 
books  have  been  written  on  the  need  to  perform 
work  processes  more  efficiently  and  effectively. 
Michael  Hammer,  a  well  known  author  in  the 
field,  has  been  encouraging  the  reengineering  of 
work,  and  one  of  his  specific  principles  is  to  use 
bar  coding  (and  other  methods)  to  collect  in¬ 
formation  once  and  at  the  source.  [1]  Hammer 
and  Champy  fmd  that  understanding  the  current 
process  is  critical  to  reengineering  but  that 
“what  people  tell  analysts,  however,  is  what 
they  think  they  should  be  doing,  what  they  hap¬ 
pen  to  remember,  or  what  they’ve  been  told  to 
say;  they  do  not  tell  what  they  actually  do. 
What  people  do  and  what  they  say  they  do  are 
almost  never  the  same.”  [2]  In  several  recent 
management  studies,  we  noted  that  Federal 
agencies  must  understand  their  work  process 
flows  better  in  order  to  streamline  them.  We 
have  applied  bar  coding  as  a  tool  to  define  the 
existing  process  more  accurately. 

Although  understanding  the  process  is  the 
required  underpinning  for  changing  it,  develop¬ 
ing  a  work  process  flow  and  obtaining  timing 
data  on  the  completion  of  each  step  is  an  elabo¬ 
rate,  time-consuming,  and  resource-demanding 
process.  Managers  often  forego  developing 
work  process  flows  for  administrative  processes 
because  of  the  difficulty  in  obtaining  the  re¬ 
quired  information.  Therefore,  if  we  could  de¬ 
vise  a  method  for  developing  accurate  work 
process  flows  more  readily,  managers  would 
probably  make  greater  use  of  them. 

Using  bar  codes  is  one  means  for  obtaining 
accurate  information  easily  to  aid  in  developing 
process  flow  information  and  for  business 
reengineering  and  streamlining.  Bar  codes  al¬ 
low  managers  to  identify  documents,  track 
them,  measure  performance,  and  report  with 
ease.  The  bar  code  report  can  be  designed  so 
that  it  highlights  or  sets  a  flag  to  indicate  the 
need  for  an  action.  To  be  most  effective,  bar 
coding  must  be  integrated  with  performance 
measures,  whose  standards  can  be  built  into  bar 
code  management  reports. 


Important  advantages  gained  from  the  use 
of  bar  code  technology  are  accuracy  and  speed, 
both  of  which  contribute  to  cost-effectiveness. 
[3]  When  a  data  collection  task  has  been  per¬ 
formed  using  bar  code  scanners  and  laser 
wands,  the  data  collection  time  is  often  reduced 
to  40-50  percent  of  that  required  for  manual 
collection.  In  addition  to  saving  time,  bar  cod¬ 
ing  eliminates  errors.  In  bar  code  technology, 
“what  you  read,  is  what  you  get.” 

Our  research  to-date  has  been  limited  to 
the  application  of  bar  coding  to  the  procurement 
process.  We  believe  that  it  has  similar  applica¬ 
tion  to  such  other  administrative  areas  as  con¬ 
tract  planning,  budgeting,  and  grants  processing 
which  may  be  subjects  for  future  research. 

HISTORY  OF  BAR  CODES 

The  history  of  bar  codes  presented  in  this 
section  is  excerpted  from  Roger  Palmer’s  book. 
The  Bar  Code  Book.  [4] 

First  patented  in  1949,  bar  codes  are  a  se¬ 
ries  of  wide  and  narrow  black  “bars”  and  white 
spaces  printed  on  a  label.  They  are  now  widely 
recognized  as  the  universal  product  codes 
(UPCs)  read  by  scanners  in  retail  stores.  Bar 
code  readers  are  optical  devices  for  scanning 
bar  code  labels  with  a  beam  of  light  and  con¬ 
verting  the  “on  and  off’  light  reflections  to 
digital  data.  Since  the  early  1980s,  bar  code 
symbologies  have  been  established  as  standard¬ 
ized  codes  that  define  letters,  numerals,  and 
symbols  in  terms  of  “bar”  and  space  combina¬ 
tions  on  the  bar  code  label.  One  common  stan¬ 
dard —  the  Federal  Standard  Bar  Code 
(Code  39)  —  can  interpret  128  different  combi¬ 
nations  of  bars  and  spaces  on  the  bar  code  label. 
These  combinations  can  include  the  upper  and 
lower  case  alphabet,  numerals  0  to  9,  plus  vari¬ 
ous  symbols  and  characters.  Currently,  efforts 
are  directed  at  achieving  higher  densities  of 
coded  data  on  the  bar  code  label. 
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The  desire  to  automate  supermarket 
checkouts  drove  the  early  technological  devel¬ 
opment  of  bar  coding.  Wallace  Flint,  son  of  a 
Massachusetts  grocery  wholesaler,  wrote  his 
1932  master’s  thesis  at  Harvard  on  a  system  for 
automating  supermarket  checkout  counters. 
Flint’s  proposed  system  used  flow  racks  and 
punched  cards  to  dispense  products  to  custom¬ 
ers  automatically.  The  proposal  was  economi¬ 
cally  unfeasible,  but  for  the  first  time,  the 
benefits  of  an  automated  checkout  had  been 
completely  documented.  Forty  years  later,  Flint 
was  the  vice  president  of  the  National  Associa¬ 
tion  of  Food  Chains,  and  he  actively  supported 
the  standardization  effort  that  led  to  the  Univer¬ 
sal  Product  Code. 

In  the  late  1940s,  Joe  Woodland  and  Bemy 
Silver  were  investigating  technical  approaches 
that  would  allow  prices  of  grocery  items  to  be 
read  automatically  at  the  checkout  stand.  Sev¬ 
eral  approaches  were  pursued,  and  their  devel¬ 
opments  culminated  in  the  filing  of  U.S.  Patent 
2,612,994  in  1949.  Their  patent  describes  a  cir¬ 
cular  printed  pattern  that  resembles  a  miniature 
archery  target.  The  concentric  rings  of  the  tar¬ 
get  are,  however,  simply  bars  and  spaces  curved 
into  a  circular  form.  Conceptually,  bull’s-eye 
coding  and  bar  coding  are  the  same. 

Serious  efforts  toward  developing  a  stan¬ 
dard  for  automating  the  supermarket  point-of- 
sale  began  in  1968.  RCA  developed  a  bull’s- 
eye  symbol  and  scanner  that  operated  in  a 
Kroger  grocery  store  in  Cincinnati  for  an  18- 
month  period  beginning  in  1972.  This  test  store 
provided  much  valuable  data  for  cost-benefit 
analysis  and  system  refinement.  Meanwhile,  in 
mid- 1970  a  grocery  industry  ad  hoc  committee 
was  formed.  A  massive  symbol  evaluation  was 
undertaken  that  included  laboratory  tests  by 
Battelle  Memorial  Institute,  printing  tolerance 
tests  assisted  by  the  Graphic  Arts  Technical 
Foundation,  print  ability  tests  by  participating 
grocery  manufacturers,  and  store  tests  of  com¬ 
plete  working  systems.  This  effort  concluded 
with  selection  of  the  UPC  symbol  as  the  indus¬ 


try  standard  on  April  3, 1973.  The  early  success 
of  UPC  in  U.S.  and  Canadian  supermarkets  en¬ 
couraged  foreign,  particularly  European,  inter¬ 
est  in  the  system.  This  interest  led  to  the  adop¬ 
tion  of  the  European  Article  Numbering  (EAN) 
code  and  symbol  in  December  1976. 

Industrial  applications  of  bar  codes  can  be 
traced  to  the  early  1960s.  In  1962  E.F.  Brinker 
of  Westinghouse  Air  Brake  filed  a  patent  that 
described  a  bar  code  attached  to  the  side  of  a 
railroad  car.  Several  companies  pursued  ware¬ 
house  and  related  applications  following  the 
initial  railroad  experience.  In  1971  the  Plessey 
Company  developed  a  bar  code  and  reading 
system  for  library  checkout.  Codabar  was  de¬ 
veloped  by  Monarch  Marking  Systems  in  1972. 
These  symbologies  continue  to  be  used  in  li¬ 
braries  and  in  blood  collection  applications. 

During  the  1970s,  bar  coding  became  in¬ 
creasingly  practical  and  economical  with  the 
advent  of  low-cost  electronics  —  microproces¬ 
sors  in  particular  —  and  the  availability  of 
smaller,  lower  cost  lasers.  As  bar  codes  became 
technically  and  economically  more  viable  dur¬ 
ing  the  1970s,  the  UPC  retail  system  provided 
stability  and  acted  as  a  stimulus  for  industrial 
acceptance.  In  the  late  1980s,  refinement  efforts 
were  concentrated  on  nonconventional  sym¬ 
bology  that  offers  significant  density  advan¬ 
tages  over  more  traditional  bar  codes. 

APPLICATION  OF  BAR  CODES 

The  use  of  bar  code  technology  is  limited 
only  by  the  creative  talents  and  initiative  avail¬ 
able  to  find  its  many  applications.  Its  use  can 
attain  several  objectives  simultaneously.  First, 
it  is  an  excellent  tool  to  use  in  establishing  and 
validating  work  process  flows.  Second,  it  is  an 
effective  tool  for  aiding  management  in  gather¬ 
ing  and  analyzing  data.  Third,  it  provides  a  fail¬ 
safe  mechanism  for  tracking  the  status  and  lo¬ 
cation  of  all  documents  within  an  organization. 

Data  from  bar  codes  can  be  obtained 
through  the  use  of  electronic  scanning  readers 
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rather  than  by  keyboard  or  written  entry,  thus 
eliminating  visual,  typing,  or  other  errors.  Fur¬ 
ther,  data  entry  with  a  scanner  requires  only  a 
fraction  of  a  second.  The  software  used  with 
bar  coding  also  provides  great  flexibility  by 
enabling  users  to  change  some  or  all  of  the  re¬ 
corded  characteristics  of  a  commodity  or  proc¬ 
ess.  It  does  so  by  changing  the  record  within 
the  computer  data  base  itself  rather  than  by 
changing  the  affixed  bar  code  label.  For  ex¬ 
ample,  grocery  stores  often  change  the  price  of 
commodities  on  display.  If  the  grocer  wishes  to 
change  the  price  of  a  can  of  soup  from  $.79  to 
$.59  cents  for  a  special  sale  the  bar  code  label 
remains  on  the  can  of  soup,  to  identify  it  as  a 
unique  item,  and  the  computer  data  base  is  re¬ 
vised  to  reflect  the  price  change.  When  the 
customer  arrives  at  the  check-out  counter  with 
the  can  of  soup,  the  cashier  scans  the  bar  code, 
that  information  is  automatically  fed  into  the 
computer,  and  the  revised  price  of  $.59  cents 
appears  on  the  cashier’s  display.  The  cashier 
does  not  need  any  technical  knowledge  of  the 
commodity  being  scanned.  The  bar  code  label 
and  the  computer  data  base  together  provide  the 
required  information. 

We  developed  and  tested  the  use  of  bar 
codes  so  that  we  could  fashion  a  credible 
tracking,  measurement,  and  management  tool 
that  had  many  different  administrative  applica¬ 
tions  for  potential  users. 

PROCESS  FLOW 

General 

The  first  step  in  studying  a  process  is  to 
have  the  manager  identify  the  types  of  actions 
that  such  a  study  can  assist,  e.g.,  procurements 
that  require  source  selection,  vendor  deliveries, 
and  invoices.  The  manager  must  then  decide  on 
a  time  period  during  which  a  statistically  sig¬ 
nificant  number  of  those  actions  will  likely  be 
completed. 


Government  contracting  personnel  are  fre¬ 
quently  overwhelmed  by  preaward  actions  and 
post-award  monitoring  responsibilities.  In  de¬ 
veloping  previous  acquisition  process  flows,  we 
noted  the  need  to  streamline  those  activities. 
Reducing  the  burden  of  routine  document 
handling  and  tracking  is  an  important  contribu¬ 
tion  to  both  preaward  and  postaward  activities. 
Issuing  status  reports  for  only  those  situations 
that  are  “out-of-tolerance”  can  substantially  re¬ 
duce  the  workload  of  contracting  personnel. 
Establishing  time  standards  for  completing  each 
activity  that  constitutes  a  processing  step  and 
integrating  those  standards  into  the  reports  pre¬ 
pared  from  the  bar  code  data  base  are  prereq¬ 
uisites  to  “reporting  by  exception.” 

The  following  six  activities  characterize 
the  overall  procedure  we  recommend  to  pro¬ 
duce  a  process  flow  diagram  and  analysis  for  an 
operating  component  that  has  no  current  proc¬ 
ess  flow  diagram: 

•  Discuss  the  program  and  requirements, 
including  the  goals  and  objectives, 
with  the  client. 

•  Gather  information  from  selected  client 
personnel. 

•  Draft  the  process  flow  milestone  steps 
and  develop  data  collection  forms. 

•  Collect  and  enter  the  data. 

•  Analyze  the  data  and  prepare  the  final 
process  flow  diagram. 

•  Integrate  the  results  into  a  final  report. 

LMI  Case  Study 

To  provide  a  simplified  system  to  reduce 
the  work  load  of  professional  procurement  per¬ 
sonnel  and  improve  their  effectiveness,  LMI 
undertook  an  independent  research  and  devel¬ 
opment  task  to  develop  and  test  a  tracking  sys¬ 
tem  based  on  using  bar  codes.  Our  purpose  was 
to  develop  a  bar  code  tracking  model  to  aid  in 
streamlining  and  simplifying  the  procurement 
process,  improving  the  responsiveness  of  the 
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reporting  system,  identifying  the  location  of  all 
documents  at  all  times,  providing  flags  to  indi¬ 
cate  when  an  action  must  be  taken,  and  fixing 
responsibility  for  each  step  in  the  process. 

We  developed  and  tested  a  bar  code 
tracking  system  using  LMI’s  in-house  procure¬ 
ment  office.  We  selected  that  office  because  it 
awards  and  administers  both  purchase  orders 
and  contracts  and  thus  provides  a  representative 
contracting  environment  for  the  test.  In  our  test, 
we  assessed  the  feasibility  and  cost  effective¬ 
ness  of  using  bar  coding  techniques  to  track 
documents  such  as  purchase  requests,  purchase 
orders,  subcontracts,  approval  memos,  deliver¬ 
able  reports,  and  invoices  through  all  of  the  cur¬ 
rent  administration  steps.  We  also  had  the  fol¬ 
lowing  objectives: 

•  To  establish  a  process  flow  procure¬ 
ment  model  that  can  be  readily  ex¬ 
panded  and  applied  to  the  more  com¬ 
plex  procurement  environment  of 
larger  organizations. 

•  To  develop  reporting  formats  that  can 
be  displayed  either  electronically  or  in 
hard  copy.  The  reports  will  be  based 
on  the  concept  of  reporting  by  excep¬ 
tion. 

In  the  following  paragraphs,  we  describe 
the  hardware  and  software  that  we  used,  the 
methodology  that  we  followed,  the  resultant 
products,  and  the  test  issues  and  their  resolu¬ 
tions. 

Hardware  and  Software 

The  following  hardware  and  software  is 
needed  to  implement  a  basic  bar  code  system: 

•  A  portable  bar  code  scanner  and  soft¬ 
ware  for  programming  it  with  the  de¬ 
sired  bar  code  symbology 

•  Software  for  generating  bar  code  labels 
with  the  desired  identification  infor¬ 
mation 


•  A  laser  printer  or  a  specialized  bar 
code  printer  and  labels 

•  Hardware  for  charging  the  batteries  of 
the  portable  bar  code  scanner  and  a 
cable  interface  to  a  computer 

•  Software  to  transmit  (download)  data 
from  the  scanner  to  a  personal  com¬ 
puter  (PC) 

•  Software  to  log  incoming  scanner  data, 
process  the  data  for  tracking  and  time 
lapse  measurements,  and  produce  the 
required  management  reports  either 
electronically  or  in  hard  copy. 

A  simplified  process  flow  diagram  for  our 
bar  code  tracking  procedure  is  shown  in  Fig¬ 
ure  1.  The  three  tasks  outlined  in  Step  1  of  that 
figure  must  be  completed  to  select  specific 
hardware  and  software  before  undertaking 
Steps  2  through  11.  Some  questions  that  must 
be  answered  when  considering  the  selection  of 
bar  code  symbology  and  the  document  identifi¬ 
cation  protocol  are  as  follows: 

•  Are  the  documents  that  are  being 
tracked  part  of  a  larger  system  with  a 
previously  established  bar  code  label¬ 
ing  protocol? 

•  Are  alphanumeric  rather  than  numeric 
designators  required  for  adequate 
identification? 

•  Are  printed  text  versions  of  the  bar 
code  required  on  each  label  for  visual 
inspection? 

•  Is  the  label  merely  a  tag  for  a  specific 
document,  or  must  it  contain  signifi¬ 
cant  text  and/or  tabular  information? 

•  Must  document  labels  be  generated  on 
a  unique,  ad  hoc  basis,  or  are  sequen¬ 
tial  generic  labels  sufficient  for  identi¬ 
fication? 

In  the  early  phases  of  implementing  a  bar 
code  tracking  system,  we  found  that  flexibility 
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manual  or  automatic  exception  reports 
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in  coding  the  information  in  the  individual  bar 
codes  is  useful  (requirements  for  additional  in¬ 
formation  always  seem  to  arise  during  this  pe¬ 
riod);  we  also  found  the  need  for  flexibility  in 
the  procedures  for  downloading  and  subsequent 
processing  the  scanned  data. 

Those  conditions  put  a  premium  on  the 
availability  of  a  dedicated  PC  and  bar  code  (or 
laser)  printer  for  all  the  coding  and  data  process¬ 
ing  steps.  Fortunately,  a  simple  MS/DOS  or 
Windows  computer  and  300  dot  per  inch  laser 
printer  will  operate  the  basic  software  required. 
In  addition,  even  the  simplest  of  bar  code  scan¬ 
ners  can  store  several  hundred  bar  code  scans 
before  its  data  must  be  downloaded. 

Many  combinations  of  hardware  and  soft¬ 
ware  can  be  used  with  bar  codes,  and  we  do  not 
recommend  any  particular  manufacturer’s 
equipment.  However,  in  our  test  case,  we  used 
a  portable  hand-held  bar  code  scanner  with  8K 
of  memory  (sufficient  to  store  about  500  10- 
digit  bar  code  scans).  We  coupled  the  scanner 
and  its  battery  recharger  and  data  downloader 
station  to  an  MS/DOS  PC  through  a  communi¬ 
cations  software  package.  We  selected  a  com¬ 
patible  printing  software  package  that  operated 
on  the  same  computer  system  as  the  scanner. 
The  bar  code  symbols  used  in  this  project  were 
printed  on  1”  x  2  Vi'  paper  labels  with  a  laseijet 
printer.  Although  many  scanners,  including  the 
one  selected,  can  process  a  variety  of  bar  code 
formats,  we  used  the  Federal  Standard  Code  39 


for  all  the  evaluation  tests.  Figure  2  presents  a 
summary  listing  of  the  bar  code  system  ele¬ 
ments  used  in  our  research. 

Methodology 

Our  initial  action  was  to  document  the  ac¬ 
tual  steps  in  the  procurement  process,  from  the 
initiation  of  a  purchase  request  through  contract 
award  and  administration  to  closeout.  We  iden¬ 
tified  those  individuals  playing  a  major  role  in 
the  procurement  process,  e.g.,  approving  offi¬ 
cials,  project  officers,  and  receiving,  finance, 
and  accounting  personnel,  as  “processing  ac¬ 
tivities.” 

Next  we  generated  a  bar  code  label  to  be 
affixed  to  each  new  incoming  procurement  re¬ 
quest  (PR)  and  to  be  read  by  a  bar  code  scanner 
first  when  that  request  arrives  at  a  designated 
processing  activity  and  a  second  time  when  it 
leaves  that  activity.  Each  additional  document 
generated  internally  in  support  of  the  incoming 
document  (e.g.,  statement  of  work,  evaluation 
plan)  which  is  scanned  in  and  out  of  each  sub¬ 
sequent  processing  activity,  has  a  bar  code  label 
affixed  to  it. 

Third,  we  affixed  a  separate  set  of  bar  code 
labels  to  “In”  and  “Out”  baskets  at  each  proc¬ 
essing  activity’s  location  as  a  unique  identifier 
of  each  activity  (even  though,  in  a  small  office, 
a  number  of  activities  may  actually  be  executed 
by  one  person  at  one  location).  For  example,  at 


Figure  2. 

Summary  listing  of  Bar  Code  System  Elements 


Type  of  equipment/software 

Description 

Portable  scanner 

Time  Wand  1,  8K  memory,  with  charger/downloader; 
Videx,  Inc.,  Corvallis,  Ore. 

Communications  package 

Time  Wand  1;  Videx,  Inc. 

Bar  code  printing  software 

EZBarcode  II;  Time  Keeping  Systems,  Inc.,  Cleveland, 
Ohio 

PC 

MS/DOS  or  Windows,  286  and  above 

Bar  code  printer 

HP  Laserjet  III 

Bar  code  labels 

Avery  No.  5160,  1”  x  2Vz,  paper 
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the  first  processing  activity  in  Figure  1  (Step  5), 
the  documents  courier  scans  the  in-basket  label 
(for  simplicity,  only  a  bar  code  label  called 
“Desk  1,”  “Desk  2,”  etc.,  is  shown  in  the  fig¬ 
ure).  The  document  courier  also  scans  the  out- 
basket  label  and  any  PRs  or  other  labeled 
documents  that  are  ready  to  move  to  the  next 
processing  activity. 

Fourth,  we  provided  contractors  with  a 
sheet  of  bar  code  labels  that  match  the  purchase 
order  issued  to  them.  We  requested  the  contrac¬ 
tor  to  affix  one  of  those  labels  to  each  of  their 
invoices,  shipping  documents,  deliverables,  or 
other  related  communications. 

Fifth,  we  issued  a  set  of  labels  identified  as 
“Receiving”  to  the  receiving  department  along 
with  a  copy  of  the  purchase  order.  These  labels 
were  to  be  affixed  to  incoming  contractor 
documents  that  the  contractor  failed  to  label  as 
requested.  We  issued  a  set  of  labels  identified 
as  “Accounting”  to  the  accounting  department 
along  with  a  copy  of  the  purchase  order.  The 
accounting  department  was  to  affix  those  labels 
to  the  incoming  invoice  if  the  contractor  had  not 
labeled  it  as  requested. 

Resultant  Products 

We  believe  that  the  effectiveness  of  bar 
code  technology  for  document  tracking  and  data 
base  generation  can  be  more  readily  understood 
when  we  combine  the  results  of  the  internal 
LMI  process  flow  study  of  its  procurement  ac¬ 
tivities  with  similar  studies  that  we  performed  at 
several  Federal  civil  agencies.  In  the  following 
paragraphs,  we  briefly  describe  our  process 
flow  work  and  how  we  have  integrated  bar 
coding  applications  into  it. 

A  process  flow  diagram,  in  hard  copy  or  in 
electronic  format,  is  an  evaluation  tool  that 
graphically  maps  the  activities  that  make  up  a 
process.  It  is  visual  representation  of  the  se¬ 
quential  and/or  concurrent  steps  in  a  process, 
the  interfaces  that  exist  between  organizational 


units  involved  in  the  process,  and  the  time  rela¬ 
tionships  of  one  step  to  another  in  the  process. 
In  developing  process  flow  diagrams,  LMI  gen¬ 
erates  a  series  of  linked  “activity  boxes”  that 
contain  the  following  elements  of  data  relevant 
to  each  step  in  the  process: 

•  Task  Description  —  a  brief  description 
of  the  activity  taking  place  at  that  step 

•  Task  Number  —  an  alphanumeric 
designator  for  the  task  description  that 
permits  easy  sorting  of  similar  tasks 

•  Start  Date  —  the  calendar  date  on 
which  the  specific  step  in  the  process 
begins 

•  Duration:  Work  Days  —  the  time  be¬ 
tween  the  acceptance  of  the  task  in  one 
step  and  the  handing  off  of  that  task  to 
the  next  step  in  the  process 

•  Task  Hours:  Professional/Support/ 
Contractor  —  the  actual  hands-on,  or 
applied  time  for  completion  of  the  task 
for  the  three  categories  of  labor 

•  Law,  Regulation,  Policy,  Practice  — 
the  written  or  oral  requirement  for  the 
specific  step  in  the  process. 

In  addition  to  Hammer  and  Champy  cited 
above,  other  authors  who  discuss  process  flow 
in  the  contracting  function  are  noted  in  “From 
Budget  to  Award:  Acquisition  Process  Flow 
Analysis.”[5]  Once  the  existing  process  is 
known,  we  can  begin  the  task  of  simplifying, 
streamlining,  or  business  reengineering. 

We  found  that  in  most  procurement  of¬ 
fices,  a  unique  set  of  process  flow  steps  that  fit 
the  specific  operational  requirements  have 
evolved.  We  also  found  that  while  the  number 
of  steps  and  the  language  used  to  describe  those 
steps  differ  from  organization  to  organization, 
an  underlying  set  of  steps  exists  and  can  be  used 
for  comparison  purposes.  In  this  paper  we  use 
the  “generic”  set  of  steps  we  developed  to  illus- 
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trate  bar  code  document  tracking  in  a  process 
flow  environment. 

Figure  3  identifies  the  10  generic  contract 
award  process  flow  steps,  each  represented  by  a 
desk  containing  an  “in”  box  and  an  “out”  box 
with  each  having  a  unique  bar  code  label  identi¬ 
fying  that  step.  Similarly,  each  document  gen¬ 
erated  in  support  of  the  incoming  procurement 
request  (PR1000,  CBD1000,  RFP1000,  to 
A/P1000)  contains  the  bar  code  symbols  that 
uniquely  apply  to  each  of  those  documents. 

All  bar  coded  documents  in  “play”  at  any 
time  are  monitored  on  an  agreed  schedule  — 
usually  daily  —  through  use  of  a  portable  bar 
code  scanner.  Generally,  in  a  small  contracts 
office,  one  person,  acting  as  a  courier,  is  trained 
to  operate  the  bar  code  scanner  and  is  responsi¬ 
ble  for  scanning  the  contents  of  each  “In”  and 
“Out”  box  at  each  processing  activity  location 
when  an  item  is  transferred  to  the  next  step  by 
the  courier.  The  scanned  bar  code  data  are 
downloaded  into  a  PC  where  the  data  can  be 
automatically  processed  into  various  reporting 
formats. 

From  the  processed  data,  status  reports  can 
be  tailored  to  the  needs  of  each  organization  and 
can  be  prepared  manually  or  automatically. 
Several  such  reports  are  described  below  under 
the  application  of  bar  coding  “As  an  Analytical 
Tool.” 

Issues  and  Resolutions 

In  this  section,  we  describe  some  of  the 
problems  we  encountered  during  the  LMI  case 
study  and  some  solutions  we  believe  should 
correct  those  problems. 

We  found  it  necessary  to  make  several  it¬ 
erations  in  the  contracting  office’s  work  process 
flow  before  we  could  bring  perception  into  line 
with  reality.  This  step  took  the  most  time.  Af¬ 
ter  we  revised  the  existing  process  flow,  the  rest 
of  the  process  of  establishing  the  bar  code 
tracking  system  fell  into  place.  While  the  exist¬ 


ing  process  flow  must  be  understood  prior  to 
establishing  a  bar  code  tracking  system,  the  ac¬ 
tual  implementation  of  the  tracking  system  can 
be  instrumental  in  helping  to  refine  the  accuracy 
of  the  process  flow  itself.  Steps  that  are  over¬ 
looked  as  well  as  steps  that  are  seldom  or  never 
used  become  obvious.  During  the  course  of  the 
test,  we  also  subdivided  a  few  steps  that  had 
initially  been  combined. 

The  initial  test  at  LMI  used  a  single  person 
to  perform  all  of  the  bar  code  reading  for  the 
entire  process.  That  approach,  while  time  con¬ 
suming  for  the  individual  involved  but  not  a 
severe  problem,  became  a  hindrance  during 
busy  periods.  We  estimate  that  at  a  maximum, 
the  bar  code  reading  added  two  minutes  to  some 
steps,  especially  if  the  purchasing  professional 
forgets  to  log  it  in  at  the  start  of  each  processing 
step  and  if  the  bar  code  is  not  read  on  the  day  of 
receipt;  both  problems  require  manual  correc¬ 
tions.  The  person  who  scans  the  bar  code  must 
do  so  in  a  timely  manner  in  order  to  ensure  ac¬ 
curacy  in  the  time  records.  In  a  complete  appli¬ 
cation,  however,  the  bar  code  scanning  would 
be  done  either  by  each  person  working  on  the 
document  with  her/his  own  reader  as  it  was 
processed,  or  by  a  courier  who  moved  the 
document  from  step-to-step  during  the  process. 
The  LMI  purchasing  professional  estimated  that 
a  day  or  day-and-a-half  would  be  required  to 
learn  the  system. 

Initially  two  sets  of  steps  —  one  for  pur¬ 
chase  orders  and  one  for  contracts  —  were  de¬ 
veloped.  However,  since  most  of  the  steps  were 
the  same  for  each  group,  we  decided  to  use  only 
one  set  of  22  steps.  Those  steps  comprise  both 
contract  award  and  administration.  Therefore 
they  are  different  from  the  10  generic  steps  re¬ 
ferred  to  in  Figure  3  which  contains  only  the 
contract  award  steps  and  does  not  include  ad¬ 
ministration.  During  the  test  period,  that  com¬ 
bination  of  steps  sometimes  required  the  person 
doing  the  scanning  to  pause  to  reflect  on  the 
correct  sequence  to  use  in  the  case  at  hand.  In  a 
complete  application,  the  processing  steps  for 
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Figure  3.  Generic  Contract  Award  Process  Flow  Steps  for  Bar  Code  Tracking 


each  type  of  action  would  be  separately  main¬ 
tained. 

The  need  for  revising  the  steps  in  a  com¬ 
plete  application  depends  on  the  accuracy  and 
completeness  of  the  existing  process  flow,  the 
number  of  changes  that  occur  in  the  processing 
cycle,  and  the  use  of  the  collected  information. 
The  process  flow  we  used  for  our  test  bed  did 
not  incorporate  the  steps  associated  with  devel¬ 
oping  a  request  for  quotations  for  subcontracts, 
nor  did  it  provide  steps  for  competitive  actions 
or  modifications  to  purchase  orders.  While 
their  omission  did  not  detract  from  our  test  re¬ 
sults,  in  a  complete  application  all  significant 
steps  should  be  included. 

Some  processing  actions  do  not  readily 
lend  themselves  to  recording.  For  example,  the 
furnishing  of  a  service,  such  as  for  copier  repair, 
does  not  produce  an  object  that  can  be  bar 
coded  and  scanned.  As  a  result,  the  action  will 
often  be  missed  unless  its  scanning  into  the 
system  is  simulated.  For  example,  “copier  re¬ 
pair”  could  be  recorded  by  scanning  the  bar 
code  label  from  a  master  list  of  events. 

The  purchase  order  number  is  preferred  as 
the  tracking  number.  Thus,  its  bar  code  has  to 
be  assigned  before  the  document  is  first  read 
into  the  tracking  system.  This  assignment  can 
be  made  by  obtaining  numbers  as  needed  from 
a  central  office  or  by  having  blocks  of  purchase 
order  numbers  assigned  to  offices  in  which 
documents  are  originated. 

Once  the  purchase  order  has  been  awarded 
to  a  contractor,  the  name  of  the  contractor 
should  be  entered  into  the  tracking  system, 
since  the  purchasing  office  typically  tracks  open 
actions  by  the  name  of  the  contractor. 

In  a  small  office,  the  most  practical  ap¬ 
proach  may  be  to  print  a  complete  set  of  labels 
for  each  action.  However,  when  that  approach 
is  followed,  the  unused  labels  must  be  available 
until  needed.  We  found  that  the  sheet  of  unused 
labels  could  be  retained  inside  the  purchase  or¬ 


der  folder.  In  a  larger  operation  that  can  support 
multiple  printers,  label  storage  is  not  an  issue 
because  the  labels  can  be  printed  as  the  action 
occurs.  Having  multiple  printers  and  the  ability 
to  produce  labels  on  demand  also  solves  the 
problem  of  needing  extra  labels  for  partial 
shipments  or  partial  invoices. 

Purchasing  personnel  may  be  concerned 
that  the  tracking  system  will  be  used  primarily 
for  monitoring  individual  performance  rather 
than  tracking  the  location  and  status  of  the  pur¬ 
chase  orders  and  contracts.  If  used  for  individ¬ 
ual  monitoring,  they  are  concerned  about  who 
will  determine  the  processing  time  standards 
and  how  they  will  be  applied.  By  involving 
purchasing  personnel  when  establishing  these 
standards  and  their  application,  managers  may 
help  alleviate  these  concerns. 

OTHER  APPLICATIONS  OF  BAR 
CODING 

Our  overall  purpose  in  developing  and 
testing  a  bar  coding  system  was  to  fashion  a 
credible  tracking,  measurement  and  manage¬ 
ment  tool  that  will  have  many  different  adminis¬ 
trative  applications  for  potential  users  and  to 
build  on  our  research  and  analysis  on  process 
flow  diagramming.  One  of  the  primary  benefits 
of  bar  code  implementation  is  the  creation  of  a 
data  base  that  can  be  used  to  develop  various 
management  reports  and  for  document  control. 

As  an  Analytical  Tool 

The  data  base  can  contain  whatever  ele¬ 
ments  managers  desire,  but  usually  data  from 
bar  code  scanning  contain  the  identifying  num¬ 
ber  of  the  document  and  the  dates  the  document 
entered  and  completed  each  step.  On  the  basis 
of  organizational  requirements,  users  must  es¬ 
tablish  the  types  and  categories  of  documents 
that  are  to  be  bar  coded.  The  extent  of  bar  cod¬ 
ing  application  may  depend  upon  such  consid¬ 
erations  as  the  size  of  the  office,  its  complexity, 
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the  contracting  volume,  the  number  of  indi¬ 
viduals  involved  in  the  process,  the  size  of  the 
contracting  staff,  and  the  risk  and  cost  of  lost  or 
misplaced  documents.  Once  documents  have 
been  bar  coded,  they  can  be  traced  through  the 
process. 

Other  identifying  information  may  be  use¬ 
ful  in  the  data  base;  that  information  includes 
the  originating  office,  the  dollar  value  of  the 
action,  the  type  of  item  being  purchased,  the 
name  of  the  company  that  was  awarded  the 
contract,  and  whether  the  company  is  a  small 
business  or  minority  firm. 

The  data  base,  in  part,  may  be  generated  by 
scanning  documents  as  they  move  through  the 
process.  Those  data  can  be  used  to  develop  re¬ 
ports  that  can  then  serve  as  analytical  tools.  The 
reports  only  answer  questions  on  timeliness; 
they  do  not  provide  any  indication  of  why  an 
event  may  have  occurred  when  it  did.  The  fol¬ 
lowing  types  of  reports  might  be  generated: 

•  Contract  Process  Flow  Exception  Re¬ 
port  —  Processing  Days  in  Excess  of 
Baseline  Days 

•  Contract  Process  Flow  Alert  Report  — 
Daily  “Hot  List”  Action  Report 

•  Process  Flow  Step  Backlog  Report 

•  Process  Flow  Variance  Report 

•  Process  Flow  Elapsed  Time  Per  Step 
Report 

•  Process  Flow  Completed  Action 
Report. 

The  following  paragraphs  present  a  dis¬ 
cussion  of  the  content  and  use  of  each  of  those 
six  reports. 

Contract  Process  Flow  Exception  Report 
—  Processing  Days  in  Excess  of  Baseline 
Days.  A  prototype  of  this  report  is  shown  in 
Figure  4.  Preliminary  standard  elapsed  process¬ 
ing  times  (baseline)  for  each  step  are  shown  in 


row  25  of  Figure  4.  At  the  bottom  of  Figure  4, 
for  each  processing  step,  we  also  show  a  range 
of  days  as  “Standard  Processing  Window, 
Days”  (Row  27).  The  elapsed  time  shown  is 
the  actual  average  time  taken  to  complete  each 
step  based  on  our  experience  at  a  civil  govern¬ 
ment  agency.  The  numbers  shown  in  the  blocks 
in  the  body  of  the  report  are  those  in  excess  of 
the  baseline  for  that  particular  action.  This  form 
of  report  enables  contract  professionals  to  focus 
their  attention  on  contract  actions  or  other  ad¬ 
ministrative  matters  on  an  exception  basis.  If 
actual  processing  time  is  above  the  baseline  (as 
indicated  in  Figure  4),  the  standard  processing 
time  is  exceeded,  and  immediate  corrective  ac¬ 
tion  should  be  taken.  This  method  of  progress 
monitoring  provides  various  supervisory  levels 
with  an  effective  tool  for  managing  the  various 
stages  of  the  procurement  process  or  any  other 
administrative  activity. 

Contract  Process  Flow  Alert  Report  — 
Daily  “Hot  List ”  Action  Report.  Figure  5  is  a 
prototype  of  this  report.  The  report  highlights 
for  both  the  specialist  and  the  supervisor  that  an 
action  is  approaching  the  processing  time  stan¬ 
dard.  Its  purpose  is  to  provide  a  daily  alert  that 
a  contract  action  is  coming  due  by  flagging  it 
before  it  reaches  the  end  of  the  standard  time 
specified  for  it.  When  an  action  has  been  in  a 
step  for  75  percent  of  the  time  specified  as  the 
standard  for  that  step,  it  is  flagged  as  a  “hot  list” 
action  item.  The  report  then  displays  the  total 
elapsed  time  for  that  step.  We  recommend  that 
responsible  individuals  view  this  report  at  the 
beginning  of  each  working  day.  Having  this 
information  will  permit  them  to  take  appropriate 
action  before  the  standard  is  exceeded. 

Process  Flow  Step  Backlog  Report.  This 
report,  which  can  be  produced  in  several  differ¬ 
ent  formats,  shows  the  number  of  contract  ac¬ 
tions  in  each  processing  step  as  of  the  date  of 
the  report.  For  an  individual  contract  specialist, 
the  report  shows  the  current  step  of  each  action 
assigned  to  that  specialist.  A  manager  can  view 
the  workload  of  an  individual,  compare  work- 
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loads  among  specialists  or  for  the  office  as  a 
whole,  and  view  a  summary  of  the  workload 
and  current  process  step  for  each  contract  ac¬ 
tion. 

Process  Flow  Variance  Report.  This  re¬ 
port  shows  the  actual  processing  time  of  each 
step  for  each  contract  action  and  the  amount  of 
variance  from  the  baseline  standard.  Those 
times  that  are  either  above  or  below  the  stan¬ 
dard  by  more  than  10  percent  are  highlighted  in 
bold.  With  that  information,  managers  can  then 
look  for  the  common  elements  that  such  actions 
might  have  —  for  example,  they  may  be  more 
complex,  larger  than  average,  processed  from 
one  office  or  by  the  same  person,  etc.  —  and 
take  appropriate  action  in  areas  that  offer  the 
highest  potential  for  streamlining  the  process. 

Process  Flow  Elapsed  Time  Per  Step  Re¬ 
port.  This  report  is  similar  to  the  Process  Flow 
Variance  Report  except  that  it  shows  total 
elapsed  time  for  each  step  rather  than  the  vari¬ 
ance  from  the  baseline.  Some  managers  may 
find  this  report  more  useful  in  reviewing  the 
validity  of  the  baseline  time  standards  for  each 
process  step. 

One  of  the  key  performance  measures 
commonly  used  in  procurement  processing 
systems  is  the  administrative  lead  time,  that  is, 
the  actual  amount  of  time  it  takes  between  re¬ 
ceipt  of  the  procurement  request  and  contract 
award.  The  data  base  generated  by  the  bar 
coded  documents  provides  a  readily  available 
source  of  elapsed  time  measurements  for  man¬ 
agers.  They  can  study  overall  processing  time 
or  the  time  it  took  each  document  (or  all  docu¬ 
ments  or  the  average  document)  to  clear  each 
step  or  a  particular  step. 

Process  Flow  Completed  Action  Report. 
This  report  provides  an  historical  record  of  all 
contract  actions  processed  by  the  contracting 
office  during  a  specified  period  of  time.  For 
those  actions,  it  shows  the  actual  number  of 
days  that  were  spent  in  each  step  and  the  total 
amount  of  elapsed  time  for  each  between  re¬ 


ceipt  of  the  purchase  request  in  the  contracting 
office  and  contract  award. 

By  using  the  bar  code  tracking  data  de¬ 
scribed  above  and  depending  on  the  identifying 
information  that  managers  decide  to  incorporate 
into  the  data  base,  other  reports  can  also  be  pre¬ 
pared.  Examples  of  some  other  reports  that  can 
be  generated  from  this  data  base  are: 

•  Small/Disadvantaged  Business  Action 
Report 

•  Competition  Report 

•  Reports  on  Process  Timing  by  Origi¬ 
nating  Office,  Size  of  Action,  Com¬ 
plexity,  etc. 

Bar  codes  are  most  easily  used  to  collect 
data  on  the  elapsed  time  between  steps.  In 
some  cases  it  may  be  possible  to  collect  infor¬ 
mation  on  the  hands-on  time  that  is  spent  to 
complete  the  action.  Those  data  will  be  avail¬ 
able  if  each  person  working  on  the  document 
uses  the  bar  code  scanner  to  log  the  time  of 
starting  and  the  time  of  completion  (or  interrup¬ 
tion)  for  that  work  session. 

Usually,  comparing  an  office  with  itself 
over  time  provides  more  useful  data  than  com¬ 
paring  it  with  other  offices.  Such  self¬ 
comparisons  eliminate  differences  in  workload, 
procedures,  equipment,  etc.,  that  may  affect 
processing  times,  particularly  when  such  differ¬ 
ences  are  not  well  understood  by  workers,  of¬ 
fice  managers,  or  higher  level  managers. 

Application  of  Bar  Coding  to 
Document  Control 

In  addition  to  using  bar  code  technology  to 
assist  in  developing  process  flows  and  as  a 
management  analysis  tool,  such  technology  is 
easily  used  to  assist  in  document  control.  Dur¬ 
ing  the  several-month  period  involved  in  the 
contract  award  process,  managers  often  lose 
track  of  where  the  pending  action  is  in  the  proc¬ 
ess.  When  follow-up  requests  are  made,  it  is 
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often  time  consuming  to  identify  the  exact  loca¬ 
tion  and  status  of  a  particular  action.  Bar  cod¬ 
ing  provides  a  better  way  of  maintaining  control 
of  in-process  acquisition  actions. 

As  previously  mentioned,  the  data  base  can 
contain  whatever  elements  managers  desire,  but 
they  must  establish  the  types  and  categories  of 
documents  that  are  to  be  bar  coded.  Again,  the 
extent  of  bar  coding  application  depends  upon 
the  size  of  the  office,  its  complexity,  the  con¬ 
tracting  volume,  the  number  of  individuals  in¬ 
volved  in  the  process,  the  size  of  the  contracting 
staff,  and  the  risk  and  cost  of  lost  or  misplaced 
documents.  Once  the  documents  have  been  bar 
coded  they  can  be  traced  through  the  process. 

Since  a  document’s  bar  code  is  scanned 
each  time  it  moves  from  one  process  step  to 
another,  its  location  is  readily  available  in  the 
data  base.  Should  one  scanning  be  overlooked, 
the  subsequent  scan  will  identify  its  current  lo¬ 
cation.  Thus,  any  time  information  on  the  cur¬ 
rent  location  of  a  particular  document  is  desired, 
it  can  be  easily  retrieved  from  the  data  base  by 
using  the  Process  Flow  Document  Location 
Report.  Upon  entry  of  a  bar  code  or  other 
identification  of  a  document,  this  report  will 
display  the  current  location  of  the  document. 

A  manager  may  also  use  the  data  base  in¬ 
formation  to  check  on  how  many  documents  are 
at  a  particular  location.  By  early  identification 
of  bottlenecks,  managers  can  direct  resources  to 
correct  the  situation. 

CONCLUSIONS 

In  this  research  effort  we  applied  bar  code 
technology  to  the  contract  award  and  admini¬ 
stration  process.  The  premise  of  our  research 
was  that  such  technology  had  served  the  users 
in  commercial  applications  well  and  could 
similarly  be  a  useful  tool  in  analyzing  the  inter¬ 
nal  administrative  operations  of  government. 
We  found  the  technology  to  be  useful  in  process 
flow  diagramming  and  analysis,  as  an  analytical 


tool,  and  for  document  control.  In  addition,  we 
draw  the  following  conclusions: 

•  The  use  of  bar  codes  is  an  effective 
tool  for  validating  process  flows. 
Some  basic  information  must  be  used 
as  the  starting  point  in  documenting 
process  flows.  Since  “What  people  do 
and  what  they  say  they  do  are  almost 
never  the  same,”  [2]  the  discipline  and 
accountability  that  bar  code  technology 
imposes  in  tracking  documents 
through  each  process  step  into  the  data 
base  quickly  reveals  to  the  work  force 
the  steps  that  were  overlooked  when 
the  process  was  initially  described.  It 
also  highlights  that  some  steps  are  no 
longer  valid  from  the  initial  process 
flow  because  bar  code  data  cannot  be 
collected  on  steps  that  are  no  longer 
operational. 

•  Bar  codes  provide  the  means  to  easily 
develop  the  data  base  needed  to  gen¬ 
erate  desired  management  reports  on 
the  elapsed  time  spent  in  any  step  in 
the  process,  either  for  individual  ac¬ 
tions  or  for  a  group  average.  The  data 
can  also  be  used  to  flag  both  “hot  fist” 
action  items  and  overdue  actions  or 
other  variations  from  standard  process¬ 
ing  times. 

•  Bar  codes  enable  managers  to  know  at 
any  time  the  location  of  each  action 
under  their  responsibility.  This  benefit 
alone  avoids  the  periodic  major 
“search  for  the  lost  document”  activity 
that  wastes  so  many  staff  hours. 

Having  successfully  implemented  bar  code 
tracking  in  a  small  test  bed  operation,  the  next 
step  is  to  evaluate  and  document  both  contract 
award  and  postaward  contract  administration 
processing  steps  in  a  large  organization.  This 
test  will  include  establishing  step-to-step  proc¬ 
essing  time  standards  from  a  selected  sample  of 
actions.  We  can  measure  each  new  transaction 
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against  the  established  standards  and  generate 
timely  reports  for  those  actions  that  are  at  or 
approaching  out-of-tolerance  status.  We  can 
also  maintain  a  near-real-time  record  of  each 
document’s  location  in  the  process  flow. 

As  a  result  of  our  research  efforts,  we  have 
shown  both  the  feasibility  and  benefit  of  using 
bar  coding  technology  as  an  effective  manage¬ 
ment  tool  either  for  validating  process  flows 
where  they  exist  or  diagramming  them  where 
they  have  not  been  previously  documented.  It 
also  serves  as  an  effective  management  tool  and 
provides  managers  with  a  capability  for  easily 
locating  documents.  Now  that  we  have  dem¬ 
onstrated  the  applicability  of  bar  code  technol¬ 
ogy  in  one  administrative  process  at  one  loca¬ 
tion,  we  are  confident  that  it  can  also  be  applied 
to  other  similar  types  of  administrative  proc¬ 
esses  as  well.  We  intend  to  expand  our  research 
efforts  along  such  lines. 
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Forrest  Gale  and  William  McGovern 
DEFENSE  SYSTEMS  MANAGEMENT  COLLEGE 


ABSTRACT 

This  paper  presents  the  results 
of  research  into  the 
development  and  application  of 
a  computer-based  aid  to  the 
contract  negotiation  team 
process.  The  tool  that  is 
presented  by  the  authors  is 
specifically  designed  for  the 
first  phase  of  the  negotiation 
process,  wherein  the  focus  is 
upon  fact  finding  and  on 
setting  negotiation  strategy. 
It  is  in  the  fact  finding  phase 
that  information  is  obtained 
which  provides  the  underpinning 
for  susequent  development  of 
contract  negotiation  strategy 
and  the  ensuing  prosecution  of 
that  strategy.  Reported  in  the 
paper  are  preliminary  research 
results  obtained  from  student 
negotiating  team  utilization  of 
the  subject  tool  in  a  complex 
multi-attribute  negotiation 
simulation  conducted  a  number 
of  times  during  the  past  three 
years.  The  results  to  date 
indicate  a  substantial 
improvement  in:  (1)  the 
individual  negotiant's  early 
perception  of  the  structure  of 
the  negotiation  armature;  (2) 
the  early  apprehension  of  the 
critical  as  opposed  to  the 
minor  negotiation  issues;  (3) 
the  the  nature  and  orientation 
of  negotiating  team  posture  on 
critical  issues  vis-a-vis 
individual  team  member 
alignments;  and  (4)  individual 
understanding  of  the 


negotiation  force  field  and  its 
relationship  to  the  negotiation 
clock  and  the  associated  issue¬ 
time  probabilities.  A 
preliminary  finding  is  that 
these  improvements  lead  to  a 
decrease  in  the  time  it  takes 
teams  to  negotiate  complex 
issues,  coupled  with  a 
corresponding  increase  in  the 
Post-Negotiation  Satisfaction 
Scale  ratings  as  expressed  by 
the  participating  negotiants. 

If  this  paper  is  approved  for 
formal  presentation,  it  is  the 
intent  of  the  authors  to 
demonstrate  the  tool  in  the 
context  of  a  mini-negotiation 
simulation  offered  for 
attendees  during  the  paper 
presentation  session. 


INTRODUCTION 

This  tool  falls  into  the 
category  or  class  of  aids  that 
map  the  collective  perceptions 
of  task  teams  as  they  work  to 
attain  performance  objectives. 
These  tools  generally  represent 
individual  as  well  as  summative 
understandings,  and  are  usually 
designed  to  aid  teams  in 
reaching  concensus  and  in 
subsequently  developing 
performance  strategies  that 
will  be  both  individually  and 
collectively  pursued.  Tools  in 
this  group  tool  class  nearly 
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always  present  information  on 
the  relative  alignment  of  the 
team  with  respect  to  perceived 
variables  of  interest. 
Presentations  of  this  nature 
catalyze  group  exploration  of 
differences  in  alignment  and 
create  a  baseline  which  is 
frequently  a  condition 
precedent  to  increasing  group 
alignment  and,  accordingly, 
cohesion. 

The  tool  developed  during  this 
research  is  unique  in  that  it 
also  provides  a  dimensional 
comparison  that  correlates 
directly  with  (1)  the 
perceptions  of  the  same 
variables  by  another 
(competing)  group;  and  (2)  a 
perceptual  profile  whose 
patterns  and  clustering 
relationships  provide 
indicators  that  point  directly 
to  the  "most  likely"  psycho¬ 
social  path  of  the  subsequent 
negotiation,  the  allocation  of 
time  afforded  the  various 
negotiated  issues  by  the 
negotiants  of  both  teams,  and 
the  issue  priorities  that  will 
susequently  focus  and  drive  the 
negotiation  process. 


TOOL  APPLICATION 

A  graphic  representation  of 
tools  outcomes  is  shown  in 
figure  1.  The  research 
protocol  has  been  as  follows: 
negotiant  teams  come  together 
in  a  group  systems  environment 
after  several  hours  of 
collecting  data  in  fact 
finding,  but  prior  to  the 
opening  negotiation  session. 
The  teams  are  generally 
composed  of  defense  industry 
and  Department  of  Defense  mid- 
career  acquisition 


professionals,  some  of  whom 
have  one  or  more  years  of 
defense  acquisition  negotiating 
experience.  The  team  members 
have  never  previously  been  a 
member  of  the  same  negotiating 
team  and  thus  are  newly 
associated  with  each  other  in  a 
negotiation  team  environment. 
Each  team — in  a  separate 
session  —  is  asked  to 
collectively  decide  what 
factors  or  issues  are  to  be 
negotiated.  In  point  of  fact, 
the  primary  issues  are  given  to 
each  negotiant  as  part  of  the 
negotiation  simulation  process 
(they  are  directed  to  negotiate 
a  certain  set  of  issues  by 
higher  authority) .  However, 
the  simulation  allows  minor 
negotiation  issues  to  be  added 
as  part  of  the  team-developed 
negotiation  strategy.  Each 
individual  team  member,  as 
hosted  by  the  group  system 
hardware  and  software,  is  then 
invited  to  rate  on  a  scale  of  1 
to  10  (10  =  Most  Important)  the 
Importance  of  each  identified 
Negotiation  Factor  (issue)  to: 
(1)  the  individual  making  the 
rating;  and  (2)  the  other 

(opposing)  negotiating  team.  A 
similar  rating  using  a 

comparable  scale  (10  =  Most 

Flexible)  is  then  used  by  the 
individual  negotiation  team 
rater  to  rate  the  same 

Negotiation  Factors  (issues) 
in  terms  of  the  perceived 
Flexibility  of  position  of:  (1) 
the  individual/ team  of  the 
rater ;  and  ( 2 )  the  other 

(opposing)  negotiating  team. 
Finally,  a  determination  of 
"The  Likelihood  Of  Reaching 
Agreement"  is  made  by  each 
negotiant  using  a  linear 
probability  scaling  gadget  (10 
=  Certain/Highly  Likely,  1  = 
Highly  Unlikely /Won't  Happen) 
embedded  in  the  group  systems 
software.  The  results  are 
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aggregated  by  the  group  systems 
software  and  presented  to  the 
group  on  a  common  screen 
similar  to  figure  1.  A 
discussion  of  the  significance 
of  the  clustering  patterns  of 
negotiation  factor  perceptions 
and  their  relationships  and 
relative  positioning  then 
ensues,  facilitated  by  the 
researchers.  Implications  of 
the  outcomes  on  negotiation 
strategy  are  highlighted,  and 
the  teams  discuss  and  realign 
their  strategy  based  on  the 
findings  and  implications  the 
tool  application  outcomes 
provide.  This  process  is  then 
repeated  by  the  other 
(opposing)  negotiating  team 
members . 

The  negotiants  then — after  a 
suitable  interval--come 
together  in  a  neutral 
negotiating  space  and  negotiate 
to  consummation  (to  date,  94% 
of  all  teams  negotiating  the 
factors  in  the  simulation  have 
reached  agreement  in  the 
allocated  time)  a  5-factor 
contract  negotiation  utilizing 
the  developed  strategies  and 
the  Phased  Negotiation 
Technique.  The  total 
negotiation  process,  including 
the  fact  finding  period,  is  not 
allowed  to  exceed  eight  hours. 
Each  phase  in  the  negotiation 
is  followed  by  a  facilitated 
unpacking  exercise  in  which  a 
large  screen  video  playback  of 
the  performance  of  the  two 
teams  in  the  previous  phase  is 
observed  by  the  two  teams  and 
analyzed  for  gambits;  strategy; 
communication  via  body 
language;  micro-expressions  in 
the  face;  tone,  pace,  and 
substance  of  voice;  meta¬ 
language  (what  is  not  said)  ; 
good  negotiation  style  and 
technique;  and  other  analyses 
of  both  individual  and  group 


performance.  The  teams  then 
retire  to  separate  spaces  to 
realign  their  phase  strategy  to 
the  outcomes  of  the  previous 
phase. 

The  improvement  in  negotiating 
performance  is  substantial 
following  the  first  unpacking 
exercise,  and  continues  at  a 
monotonically  decreasing  pace 
after  each  phase  of  the 
negotiation  and  its  trailing 
unpacking  process. 

A  total  of  36  teams  have 
participated  in  the  negotiation 
scenario  since  1986.  Eight 
teams  have  participated  in  the 
scenario  aided  by  the  use  of 
the  group  systems-based 
strategic  negotiation  planning 
tool.  The  relative  performance 
of  the  negotiant  teams  in  both 
a  tool-augmentation  and  non¬ 
tool  aumentation  mode  is  shown 
in  figure  2.  Examination  of 
this  data  suggests  that  use  of 
the  tool  yields  an  average 
reduction  of  40.5%  in  the 
number  of  negotiating  sessions 
(phases)  required  to  reach 
agreement  on  the  same  set  of 
five  complex  negotiating 
factors:  (1)  price/costs;  (2) 
schedule  of  delivery  of 
multiple  items  in  multiple 
months/years;  (3)  delivered 
quantities;  (4)warranty 
provisions;  and  (5)  rights-in- 
data.  This  reduction  in  the 
number  of  discrete  negotiating 
events  is  accompanied  by  a  mean 
reduction  in  time  to  reach 
agreement  of  36.8%.  The  target 
price/cost  of  the  resultant 
agreement,  however  has  been 
found  not  to  change 
significantly  from  that 
obtained  in  the  non-tool 
augmented  mode. 

The  average  negotiated  contract 
price  is  $58.2  M  for  all  teams 
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negotiating  to  agreement,  and 
$58.5  M  for  teams  who  utilized 
the  group  systems  based 
strategic  negotiation  planning 
tool;  the  difference  is  not 
considered  to  be  statistically 
significant  for  the  indicated 
sample  size.  Finally, 
anecdotal  evidence  from  an  exit 
survey  of  negotiant 
satisfaction  with  the  outcome 
of  the  negotiation  process 
indicates  a  moderately  higher 
degree  of  satisfaction  with  the 
negotiation  for  those 
individuals/teams  who  have  used 
the  tool,  approximately  92%  of 
all  negotiants  using  the  tool 
during  the  negotiation  process 
profess  satisfaction  with  the 
negotiation  process  itself  and 
its  outcome.  This  compares 
favorably  with  a  satisfaction 
percentage  of  approximately  83% 
for  those  negotiants  not 
exposed  to  the  tool. 


TOOL  DISPLAY  INTERPRETATION 

The  interpretation  of  figure  1 
forms  a  portion  of  the 
facilitated  and  interactive 
post-application  process  that 
follows  the  group  systems  tool 
session.  The  following 
discussion  refers  to  the 
display  tableau  of  figure  1, 
the  perceptual  team  map  of 
negotiation  issues. 

The  upper  left  hand  quadrant  of 
the  display  contains  those 
negotiation  factors  that  will 
be  most  influential  in  shaping 
the  negotiation  process  and 
content.  In  classic 
negotiation  terms,  these  are 
the  potential  "sticking  points 
of  the  negotiation.  The 
factors  in  this  quadrant  are 
perceived  and  rated  by  each 
team  as  inflexible  in 


negotiating  position  and  of 
relatively  high  importance. 
Factors  in  this  quadrant  can  be 
expected  to  consume  a  major 
portion  of  the  negotiating  time 
if  certain  conditions  exist. 
Further,  factors  in  this 
quadrant  are  often  submerged 
early  in  a  negotiation,  surface 
"mid-negotiation",  and  from 
that  point  on  are  experienced 
predominately  as  the  focus  of 
the  negotiation. 

The  upper  right  hand  quadrant 
of  the  tool  display  contains 
the  next  most  influential 
negotiation  factors.  These 
factors  are  perceived  to  be  of 
relatively  high  importance  but 
the  negotiating  stance  of  the 
team  is  perceived  as  being  one 
of  relatively  high  flexibility. 
In  the  argot  of  professional 
negotiators,  these  factors  are 
called  "sweethearts",  because 
they  can  and  will  be  agreed 
upon  early  in  the  negotiation 
process,  and  provide  psychic 
motive  force  that  can 
successfully  move  a  negotiation 
through  its  early  stages  if 
they  are  present  in  the 
issue/factor  menu.  Factors  in 
this  quadrant  often  provide 
opportunities  for  establishing 
a  climate  of  success  in  a 
negotiation  if  certain 

conditions  exist. 

Factors  located  in  the  lower 
right  hand  quadrant  of  the  tool 
tableau  of  figure  1  are  the 
next  most  influential  set,  are 
perceived  to  be  of  relatively 
low  importance  by  the  rating 
negotiation  team,  and  are  seen 
to  possess  a  positional  lack  of 
flexibility.  Professional 

negotiators  call  these 
negotiation  factors  "add-ons". 
These  factors  are  often  tossed 
in  to  a  negotiating  milieu  late 
in  in  the  process  with  a  "take- 
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it-or- leave-it"  proviso,  or 
used  in  a  feinting  gambit  that 
attempts  to  give  an  incorrect 
impression  of  their  relative 
importance  mid-term  to  late- 
term  in  a  negotiation.  They 
are  primarily  used  as  the  coin 
of  psychological  equilibrium  by 
one  side  or  the  other, 
depending  on  whether  certain 
conditions  exist  or  not  in  the 
mid  to  late  stages  of  a 
negotiation.  They  are  rarely 
introduced  as  the  momentary 
focus  of  a  negotiation  early  in 
the  negotiation  process. 

Those  factors  located  in  the 
lower  right  hand  quadrant  of 
the  tool  tableau  are  perceived 
by  the  rating  negotiation  team 
to  possess  relatively  low 
importance  but  a  negotiating 
stance  of  relatively  high 
flexibility.  Professional 

negotiators  have  labeled 
factors  in  this  quadrant  set 
"sweeteners  or  leaders".  Focus 
on  these  factors  may  occur  at 
any  time  position  in  the 
negotiation  process,  depending 
on  the  negotiating  climate  and 
gestalt  of  the  individual 
negotiants  at  the  moment  of 
factor  introduction.  A  common 
experience  in  negotiation  is  to 
observe  that  these  factors  are 
introduced  early  (as  "loss 
leaders")  or  mid-term  to  late- 
term  in  a  negotiation,  in  which 
case  they  are  often  used  as 
"sweeteners"  (to  maintain  or 
attempt  to  obtain  psychological 
equilibrium  mid-term) ,  or  as 
"closers"  late-term,  in  an 
attempt  to  "seal  the  deal". 
Again,  certain  circumstances 
well  known  to  seasoned 
negotiators  mitigate  against, 
mediate,  or  prompt  their 
application  in  the  various 
phases  (phased  negotiation)  and 
stages  of  a  typical 
negotiation. 


The  simplified  interpretation 
of  the  relative  clustering  of  a 
negotiating  team's  perceptions 
of  the  flexibility-importance 
continuum  of  negotiation 
involves  factors  of  self  as 
opposed  to  that  of  the  opposing 
team,  as  follows:  (1)  Team 
factor  perceptions  of  group 
"self"  and  of  the  opposing  team 
that  lie  in  the  same  quadrant 
increase  the  probability  of 
agreement.  Those  clusterings 
of  the  same  negotiation  factor 
that  lie  in  different  quadrants 
are  a  cause  for  strategic 
concern. 

If  the  differences  are  only 
between  the  lower  quadrants, 
this  is  seen  to  inform  the 
timing  and  conditions  for 
introduction  of  these  factors. 

If  the  differences  are 
perceived  to  lie  only  between 
the  right  hand  quadrants,  this 
is  seen  to  inform  not  only 
timing  but  the  negotiating 
gambits  one  might  entertain 
with  respect  to  these  factors, 
and  the  gambits  and  timing  one 
might  expect  to  be  pursued  by 
the  other  (opposing)  team. 

If  the  single-factor 
differences  lie  in  the  left 
hand  quadrants,  this  is  seen  as 
an  indication  of  the  potential 
for  deadlock  in  the 
negotiations,  via  interaction 
on  factors  of  relatively  low 
importance  to  one  of  the 
negotiating  teams.  Thus,  this 
pattern  is  an  indication  thata 
unique  set  of  stratagems 
designed  to  break  this 
unbalanced  axis  should  be 
included  in  the  negotiation 
plan. 

If  the  factor  differences 
indicated  on  the  tool  tableau 
are  in  the  upper  quadrants , 
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this  is  seen  as  the  true  focus, 
or  armature,  of  the  negotiation 
process  (possessing  the  highest 
probability  of  deadlock  but 
with  high  importance  to  both 
sides  of  the  negotiation)  and 
requiring  the  greatest  amount 
of  planning  and  contingency 
analysis;  both  to  avoid 
deadlock  and  to  increase  after¬ 
negotiation  satisfaction  with 
both  the  outcomes  of  the 
negotiation  and  the  negotiation 
process.  It  is  postulated  that 
higher  satisfaction  in  these 
two  areas  will  increase  the 
probability  of  both:  (1) 

attainment  of  sufficient 
consideration  (economic  and 
psychological)  for  performance 
of  the  agreement  by  both  sides; 
and  (2)  after-the-negotiation 
performance  on  the  terms  of  the 
agreement  by  both  parties  to 
the  negotiation. 

The  analysis  provided  by 
application  of  the  tool  thus 
has  provided  insights  about 
timing,  focus,  emphasis, 
factor-gambit  matching,  team 
alignment  (provided  by  the 
group  systems  software 
statistical  measures  of  the 
dispersion  of  individual 
responses,  etc.),  contingency 
planning  needs,  etc.  of  the 
upcoming  negotiation.  This 
information  is  a  necessary 
precursor  to  the  deep  strategy 
development  that  is  the 
hallmark  of  world  class 
negotiation  preparation  in  any 
venue. 


CONCLUSIONS / SUMMARY 

The  tentative  conclusions  to  be 
drawn  from  the  results  obtained 
to  date  from  this  research  are 
as  follows:  (1)  Time  to 


successful  conclusion  of  a 
typical  (simulated)  complex 
team  negotiation  of  the  type 
encountered  in  defense 
acquisition  may  _  be 
significantly  reduced  if  a 
perception  mapping  tool  such  as 
that  utilized  in  this  research 
is  applied  during  the  early 
stages  of  the  negotiation 
process  (ie-in  fact  finding); 
(2)  The  number  of  negotiating 
sessions  necessary  to  reach 
agreement  in  a  negotiation  of 
the  type  referred  to  above  may 
be  significantly  reduced 
through  utilization  of  a 
perceptual  mapping  tool  early 
in  the  negotiation  process;  (3) 
negotiant  satisfaction  with 
both  the  negotiation  process 
itself  and  the  negotiated 
outcome  may  be  increased  via 
utilization  of  a  perception 
mapping  tool  such  as  that 
employed  in  this  research. 

The  defense  acquisition  process 
is  arduous  and  costly .  The 
contracting  portion  of  that 
process  is  inordinately 
expensive  and  time  consuming. 
Though  implementation  of  the 
changes  mandated  by  acquisition 
reform  initiatives  will 
certainly  remove  barriers  to 
contract  performance  on  both 
sides  of  the  negotiating  table, 
said  reforms  may  not  improve 
the  efficiency  and 
effectiveness  of  the 
contracting  process  as  much  as 
direct  improvements  in  the 
contracting  process  itself.  A 
simple  and  effective  way  to 
improve  process  performance  may 
be  the  introduction  and 
widespread  utilization  of  tools 
such  as  the  one  employed  in 
this  research. 

There  are  a  number  of  questions 
that  have  arisen  as  a  result  of 
the  work  described  in  this 
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paper  that  may  be  the  focus  of 
future  research.  One  question 
that  particularly  intrigues  the 
authors  is  the  exploration  of 
the  possibility  of  joint  team 
sessions  using  this  tool  or  one 
of  similar  design.  Would  joint 

sessions,  with  their  potential 
for  tableau  overlay  and 

simultaneous  multidimensional 
analysis  of  both  the  perceptual 
framing  (afforded  by  the 

current  tool  design)  and  the 
additional  comparison  of  other 
(opposing)  team  perceptual 
accuracy,  etc.,  further  improve 
negotiation  efficiency  and 

effectiveness  ? 

What  would  be  the  affect  on 
after-agreement  performance  ? 
Upon  negotiant  satisfaction  ? 
It  is  the  intent  of  the  authors 
that  these  and  many  other  such 
questions  be  the  subject  of 
future  research. 
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File  View  Options 
1.  Cost 


1.  Cost 

2 .  Schedule 

3 .  Quantity 

4 .  Rights-in-Data 

5 .  Warranty 


Negotiant  Identification 
Session  Help 


PERFORMANCE  METRIC  TOOL  USE 

Number  Of  Negotiating  Sessions  3.7** 

(Phases  Req'd  To  Reach  Agreement) 

Time  To  Reach  Agreement  (hours)  6.25** 

(Includes  All  Pre-Negotiation 
Activity) 

Negotiated  Contract  Price  ($M)  58.5** 

(One  Of  Five  Negotiated  Factors) 

Participant  Satisfaction  (%  Sat)  92** 

(Scale:  Sat;  Mod.  Unsat;  Unsat) 

(Satisfaction:  Process  And  Outcomes) 


NO  TOOL  USE 
5.2* 

8.55* 

58.2* 

82* 


Sample  Size:  38  Teams 
*No  Tool  Use:  30  Teams 
**Tool  Use:  8  Teams 

Figure  2 . 
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TECHNICAL  SCREENING  EXPERT  SYSTEM 

TSES 


Jane  Lange  and  Brenda  L.  Kish 


ABSTRACT 

HAZARDOUS  MATERIAL 
MANAGEMENT /ENVIRONMENTAL  ISSUES 
are  identified  in  the  Navy 
Supply  Corps  Newsletter  of 
September/October  1993  as 
PREEMINENT  ISSUES...."  an  area 
of  increasing  responsibility  for 
Supply  Corps  Of f icers . . . . we  must 
provide  the  POLICY,  TOOLS,  and 
TRAINING  which  position  us  to  be 
the  provider  of  choice  for  these 
services  to  both  the  Fleet  and 
shore  stations".  This  paper 
focuses  attention  on  the  need 
for  effective  POLICY,  TOOLS,  and 
TRAINING  necessary  to  create  a 
viable  program  to  meet  the 
challenges  mandated  by  the 
Navy's  concern  for  personnel 
safety  and  the  environment.  The 
Technical  Screening  Expert 
System  (TSES)  responds  to  this 
challenge  by  providing  a  tool 
which  assists  in  the 
implementation  of  the  Chief  of 
Naval  Operation's  (CNO's) 
Hazardous  Material  Control  and 
Management  (HMC&M)  Policies  in 
an  easy-to-use  system  which  can 
be  accessed  worldwide.  Our 
implementation  plan  includes 
installation  of  TSES  on  every 
ship  in  the  fleet  and  shore- 
based  activity,  providing 
logistics  support  in  the 
procurement,  distribution, 
maintenance,  and  replacement  of 
hazardous  material.  It  further 
relates  how  TSES  implements 
CNO's  HMC&M  policies  and 
identifies  the  environmental 


advisories,  directives, 
instructions  and  regulations 
which  are  incorporated  into 
TSES.  Furthermore,  this  paper 
explains  how  TSES  fits  into  the 
realm  of  the  HMC&M  Program. 

INTRODUCTION 

On  26  July  1971,  a 
Presidential  Executive  Order 
(E.O.  11612)  entitled 
"Occupational  Safety  and  Health 
Programs  for  Federal  Employees" 
was  signed.  This  Executive 
Order  stated  that  the  Federal 
Government,  as  the  nation's 
largest  employer,  has  a  special 
obligation  to  set  an  example  for 
safe  and  healthful  employment. 
During  the  past  10  years  and 
through  several  revisions  of 
this  first  E.O.,  the  Department 
of  Defense  (DOD)  has  issued  many 
directives  and  instructions 
relative  to  implementation  of 
the  Navy  Occupational  Safety  and 
Health  Policy. 

Consistent  with  this 
policy,  TSES  was  developed  as  a 
Naval  Supply  Systems  Command 
(NAVSUP)  Small  Business 
Innovative  Research  (SBIR) 
Project  to  assist  ship  and  shore 
activities  with  implementing  the 
CNO's  HMC&M  Program.  TSES 
provides  advisory  support  to 
technical  and  procurement 
personnel  via  a  step-by-step 
logic  guide,  and  affords  users 
requisition  guidance  and 
restrictions  applicable  to  both 
stock-numbered  and 
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PROGRAM  DESCRIPTION 


non-stock-numbered  items, 
ranging  from  hazardous  and 
non-hazardous  items  to  plastic 
alternatives  to  medical/dental 
allowances.  TSES  incorporates  6 
resident  databases  and  accesses 
one  external  to  the  TSES 
software  package.  TSES  also  has 
the  capability  to  link  with 
other  data  information  modules, 
and  to  automatically  generate 
letters  and  forms  relevant  to 
the  Hazardous  Materials  Program. 

TSES  incorporates  program 
modifications  based  on  user 
experience  and  feedback  from 
on-site  demonstrations  and 
training,  not  only  in  the  use  of 
the  system,  but  in  environmental 
policy,  regulations,  and 
restrictions.  Since  its 
inception,  over  200  system 
enhancements  have  been 
integrated.  TSES  is  proving  to 
be  a  dynamic,  flexible,  and 
viable  tool  for  the  requisition 
processor . 


The  following  chart,  Figure 
1,  depicts  the  path  a 
requisition  follows  as  it  is 
processed  through  the  TSES 
software  program.  As  the  user 
inputs  data,  the  system  queries 
for  additional  information 
according  to  the  type  of 
requisitioned  material. 
Therefore,  all  questions  in 
Figure  1  are  not  applicable  to 
every  requisition.  As  the  user 
answers  questions,  the  system 
responds  with  invaluable 
advice/guidance  and 
environmental  restrictions 
applicable  to  both 
stock-numbered  and 
non-stock-numbered  items  to 
enable  the  user  to  comply  with 
the  Navy's  HMC&M  Program.  All 
acronyms  in  Figure  1  will  be 
explained  further  in  subsequent 
sections  of  this  paper. 
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TECHNICAL/PROCUREMENT  SCREENER 
INSTALL 

(requires  10  MB  of  hard  disk  space,  approximately  450 
KB  of  free  RAM,  and  DOS  3.3  or  above) 

HELP  TUTORIAL  USER'S  MANUAL 
(help  is  available  with  FI  key,  tutorial  and  user's  manual 
are  available  from  the  main  menu) 

(SHIP)  REQUISITION  (SHORE) 

TSES 

INPUT  REQUISITION,  LSN/NSN,  CAGE,  P/N,  ITEM  NAME 
SEARCHES  DATABASES 

LOCAL  NON- STD  SHML  AMAL/ADAL  AUL  PRIME 

FOUND 

PRINTS  ADVISORY  MESSAGE  AND  TECHNICAL  SCREEN  SUMMARY 

NOT  FOUND 

QUERIES  FOR  INFORMATION 

ENTER  FSC: 

HIGH  VOC/LEAD? 

(specific  FSCs) 

HAZARDOUS?  Y/N 

AUTHORIZED?  Y/N 

87  RECEIVED?  Y/N  (shore) 

(Generates  Letter/Form/Inst ruction) 

SFR  RECEIVED?  Y/N  (ship) 

(Generates  Letter/Form/ Inst ruction) 

UPDATE  LOCAL?  Y/N 

UPDATE  LTR/MSG?  Y/N 
(Generates  Letter/Form/ Instruct ion) 

PRINTS  ADVISORY  MESSAGE  AND  TECHNICAL  SCREEN  SUMMARY 
_ Figure  1  TSES  PROGRAM  DESCRIPTION _ 
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TSES  DATABASES 

Countless  hours  of  research 
are  required  to  process  a 
requisition.  The  technical/ 
procurement  screener  searches 
file  drawers  full  of  old  paper 
messages  and  pages  through 
cumbersome  technical  manuals 
that  are  sometimes  difficult  to 
read.  In  all  likelihood  these 
same  items  were  previously- 
researched  by  another  screener. 
Thanks  to  TSES,  those  days  are 
over!  Research  is  now  at  the 
screeners'  fingertips  in  the 
form  of  7  databases,  as 
explained  in  the  ensuing 
paragraphs : 

Non-Standard:  A  database 

designed  to  allow  the  TSES  user 
access  to  hazardous  and  non- 
hazardous  non- stock- numbered 
data  from  all  users.  TSES  has 
sufficient  procurement  and 
technical  data  fields  to 
accommodate  the  Inventory 
Control  Point  (ICP)  database  and 
to  be  the  vehicle  that  the  fleet 
and  shore  activities  can  use  to 
interactively  obtain  and  provide 
technical  data  during  the 
requisition  screening  process. 
The  information  is  gleaned  from 
data  sent  to  NSPCC  via  automatic 
data  input  by  users  into  their 
local  database  during  the 
screening  process.  During  this 
process,  TSES  generates  a  letter 
to  NSPCC  requesting  this  data  be 
input  into  the  Non-  Standard 
database.  This  letter/data  can 
be  printed  to  a  file  and 
disseminated  via  the  Stream¬ 
lined  Alternative  Logistics 
Transmission  System  (SALTS)/ 
MODEM/DISC.  Upon  receipt  of 
user  data,  NSPCC  verifies  the 
information,  adds  it  to  the 
Non-Standard  database  and 
redistributes  the  information  to 
all  users  on  a  quarterly  basis. 


Plastic  Removal  In  Marine 
Environment  (PRIME) :  Plastic 
and  plastic- containing  items 
create  an  environmental  hazard, 
especially  for  ships  at  sea, 
their  personnel,  and  marine 
life.  Therefore,  NAVSUP 
researches  and  maintains  a  list 
of  non-plastic  alternatives. 

This  information  is  found  in  the 
NAVSUP  "SHIP'S  GUIDE  for 
SHIPBOARD  SOLID  AND  PLASTICS 
WASTE  MANAGEMENT",  Appendix  A. 
TSES'  PRIME  database  matches 
this  data  and  displays  the  non¬ 
plastic  alternatives  upon  entry 
of  a  National  Stock  Number  (NSN) 
from  the  PRIME  database. 

Ships  Hazardous  Material  List 
(SHML) :  It  is  extremely 

important  for  ships  to  protect 
their  personnel  from  harm  caused 
by  hazardous  materials  and 
minimize  the  quantity  of 
hazardous  waste  generated  in 
day-to-day  operations. 

Therefore,  the  SHML  was 
developed  to  help  the  user 
ascertain  if  an  item  is  allowed 
onboard.  Five  (5)  APRON  codes 
comprise  the  authorization 
field.  "A"  indicates  an  item  is 
Authorized,  "P",  Prohibited, 

"R",  Restricted,  "0",  Obsolete, 
and  "N" ,  Not  Determined.  The 
SHML  also  includes  a  Replacement 
NSN  field  and  an  Ozone  Depleting 
Substance  (ODS)  field, 
identifying  items  containing 
ODS.  This  database  is  main¬ 
tained  by  the  Environmental 
Control  and  Packaging  Programs 
Division  at  NSPCC  and  is  updated 
quarterly. 

Authorized  Medical/Dental 
Allowance  Lists  (AMAL/ADAL) : 

Updated  quarterly  by  the  Naval 
Medical  Logistics  Command,  Fort 
Detrick,  MD,  these  lists 
comprise  NSNs  of  medical  items 
which  are  allowed  onboard. 
Classes  of  AMALs/ADALs, 
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described  in  the  Help  screen, 
further  delineate  these  lists. 

Hazmat  Local  Authorized  Use  List 
(AUL) :  TSES  offers,  for  the 

first  time,  a  standardized  AUL 
format  for  shore  activities. 

More  significantly,  it  provides 
a  means  of  developing  local 
HazMat  AUL  databases  and 
controlling  hazardous  material 
procurements  when  implemented  as 
part  of  the  technical  screening 
and/or  procurement  process . 

NSPCC  converts  this  data  into 
the  TSES  format  and  returns  to 
the  cognizant  user  activity. 

Local:  Maintained  by  each  user 
activity,  the  Local  database 
allows  the  user  to  take  those 
old  paper  messages  from  his  file 
drawer  and  automatically  load 
them  into  TSES  during  requisi¬ 
tion  processing.  This  saves  the 
user  countless  hours  of  search¬ 
ing'  Tor  previously  researched 
information,  permits  the  user  to 
share  data  with  other  screeners 
at  his  activity,  and  ultimately 
to  share  data  with  screeners  at 
all  user  activities  via  input 
into  the  Non-Standard  database 
by  NSPCC. 

Hazardous  Inventory  Control 
System  (HICS) :  An  item  which  is 
stocked  in  the  user's  Regional 
HAZMIN  Center  is  identified  to 
the  technical  screener  when  TSES 
finds  it  in  the  external  HICS 
database  and  refers  the 
requisition  to  the  user's  HAZMIN 
center. 

SYSTEM  CAPABILITIES/FEATURES 

LAN  Compatible:  TSES  is  local 
area  network  (LAN)  compatible, 
offering  multi-user  access  to 
databases.  This  eliminates  the 
redundancy  of  several  screeners 
researching  the  same  requisi¬ 
tion.  After  automatic  input 


from  the  requisition  screening 
menu,  all  local  users  can  share 
the  information. 

Integration  via  HotKey  User 
Menu:  TSES  offers  the  powerful, 

easy-to-use  capability  of 
integrating  external  programs, 
such  as,  Federal  Logistics  Data 
on  Compact  Disc  (FEDLOG) , 
WORDPERFECT,  Automated 
Procurement  and  Accounting  Data 
Entry  (APADE) ,  Hazardous 
Material  Information  System 
(HMIS) ,  etc.,  and  commands 
within  the  program.  Integration 
is  accomplished  via  a 
configurable,  user-defined, 
drop-down  USER  menu  and  Hot  Key 
system.  The  system  also  permits 
program  expansion  via 
integration  of  new  program 
modules  as  they  are  developed. 
Users  build  their  own  menu, 
containing  up  to  eight  external 
programs,  by  simply  selecting  a 
menu  item  and  entering  a  program 
name,  start  up  command,  and 
start  up  location  (drive  and 
directory) .  After  a  menu  item 
is  configured,  the  external 
program  is  executed  by  selecting 
the  appropriate  menu  option,  or 
pressing  the  associated  hot  key 
from  any  TSES  screen.  The 
user-defined  menu  is  also 

«"»  n  Vv  1  -f-  v*  — h  -v»  *  rT.rV\  v*  /-\  -I  IT*  O  "C1  O 

C.CDD _l JC/_L  tt:  i_  jl  will  au_y  wiicx  tr  xx i  rufnx 

via  the  Alt- FI 0  key.  This 
point -and -shoot  pop-up  menu 
simplifies  execution  of  external 
programs  and  seamlessly  inte¬ 
grates  TSES  into  the  user's 
software  suite. 

Automatic  Generation  of  Letters/ 
Messages/Forms/Instructions : 

TSES  reduces  administrative 
workload  by  including  a  variety 
of  correspondence,  defined  in 
the  following  paragraphs,  which 
is  automatically  produced  and 
can  be  configured  (personalized) 
to  each  activity's  needs. 
Correspondence  can  be  printed  or 
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directed  to  a  file  and  trans¬ 
mitted  via  SALTS,  modem,  hard¬ 
copy,  disc,  in  message  or  letter 
format . 

Ships  Hazardous  Material 
Feedback  Form  (SFR) .  An  SFR 
form  is  required  for  purchase  of 
Non-Standard  Hazardous  Material 
(HM) ,  not  identified  in  the  SHML 
as  "authorized  for  shipboard 
use",  and  must  be  signed  by  the 
Commanding  Officer  (CO)  of  the 
ship.  COMNAVSUPSYSCOM  MSG 
052345Z  August  1991  requiring 
the  CO  approval,  is  referenced 
in  a  letter/message  which  is 
automatically  generated  for 
return  to  the  requisitioner . 

NAVSUP  Form  87  Request  for 
National  Stock  Number  Assign¬ 
ment.  Just  as  the  SFR  is 
generated  for  a  ship  requisi¬ 
tion,  this  form  is  generated  for 
purchase  of  Non-Standard,  HM  for 
shore  requisitions.  A  letter/ 
message,  referencing  NAVSUPINST 
4410.50  B  Elimination  of  Local 
Stock  Numbers  (LSN)  of  03  March 
1993  is  on-line  and 
automatically  generated. 

Non-Standard  Database 
Update  Letter.  After  input  of  a 
new  item  into  the  Local 
database,  a  letter/message  which 
captures  the  data  from  the 
Screening  process,  is 
automatically  generated  to  NSPCC 
for  entry  into  the  Non-Standard 
database  and  distribution  to  all 
users  the  following  quarter. 

HazMat  (HM)  Coordinator 
Authorization  Letter.  Input  of 
a  shore  based  requisition  HM  NSN 
which  is  not  in  the  Local  HazMat 
AUL  will  produce  a  letter/ 
message  which  requires  the 
signature  of  the  HM  coordinator 
prior  to  purchase. 


Suspense  Tickler  File: 

Automatically  keeps  track  of 
dates  SFR  and  NAVSUP  87  Forms 
are  due  and  indicates  SFR  DUE 
TODAY  on  the  Screening  Menu  when 
applicable . 

Built-in  Tools:  One  of  TSES 
menus,  TOOLS,  includes  the 
previously  mentioned  letters/ 
forms,  as  well  as,  other 
significant  reference  documents, 
identified  below: 

FED-STD-313C,  Tables  I  and 
II  (Appendix  A) ,  Material  Safety 
Data,  Transportation  Data  and 
Disposal  Data  for  Hazardous 
Materials  Furnished  to  Govern¬ 
ment  Activities  of  01  Mar  88. 
Accessed  from  the  Screening  or 
Tools  Menu,  Fed-Std-313C  helps 
the  screener  identify  those 
items  which  contain  chemicals 
and  require  a  Material  Safety 
Data  Sheet  (MSDS) . 

Federal  Supply  Classifi¬ 
cation  (FSC)  Catalog,  H2-1,  Part 
1,  Groups  and  Classes  of  Sept 
94.  During  the  normal  process  of 
screening  requisitions,  the 
screener  attempts  to  match  an 
FSC  to  the  non-standard  item. 

The  entire  list  of  FSCs  is 
available  from  the  Screening  or 
Tools  Menu  to  eliminate  the  need 
for  hard  copy. 

NAVSUPINST  4200.85A,  Enel. 

3  Shore  and  Fleet  Small  Purchase 
and  Other  Simplified  Purchase 
Procedures  of  17  Sep  91.  Certain 
items,  such  as,  filing 
equipment,  require  special 
attention.  Specific  FSCs  have 
been  assigned  to  these  items 
and,  if  input,  will  automatic¬ 
ally  display  a  window  explaining 
these  requirements . 

Procurement  Related  Information: 

As  a  requisition  is  processed, 
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it  passes  through  the  hands  of 
technical  and  procurement 
personnel.  If  each  screener  is 
using  TSES,  they  have  access  to 
the  same  information  via  a 
Technical  Screen  Summary.  These 
summaries  can  be  printed  to  a 
file,  a  printer  or  the  screen. 
Each  summary  contains  all  the 
pertinent  information  input  by 
the  user  and  generated  by  the 
system  research  process.  As  can 
be  viewed  in  the  sample  summary 
in  Figure  2 ,  TSES  provides  the 
user  information  necessary  to 
process  the  requisition,  such 
as,  Material  Safety  Data  Sheet 
(MSDS) ,  Hazard  Warning  Label, 
and  ODS  requirements. 
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Reference  Information:  In  the 

VIEW  DATA  DROP-DOWN  MENU  of  the 
SCREENING  MENU,  TSES  contains  a 
wealth  of  reference  information, 
in  the  form  of  lists:  Acronyms, 
Dual  Cognizance  (COG)  Codes, 
Special  Material  Identification 
Codes  (SMIC) ,  Type  of  Storage 
(TOS)  Codes,  Special  Material 
Content  Codes  (SMCC) ,  Shelf/Life 
(S/L)  Codes,  Shelf  Life  Action 
Codes  (SLAC) ,  Unit  of  Issue  (UI) 
Codes,  Unit  Identification  Codes 
(UICs) ,  Hazardous 
Characteristics  Codes  (HCC) ,  and 
Acquisition  Advice  Codes  (AAC) ; 
as  well  as,  ODS  information. 

Ozone  Depleting  Substances 
(ODS) .  The  stratospheric  ozone 
layer  protects  the  earth  from 
the  penetration  of  harmful 
ultraviolet  (UV-B)  radiation. 

On  the  basis  of  scientific 
evidence,  a  national  and 
international  consensus  exists 
that  certain  man-made 
halocarbons,  including 
chlorof luorocarbons  (CFCs) , 
halons,  carbon  tetrachloride, 
and  methyl  chloroform,  must  be 
restricted. 

In  accordance  with  the 
Defense  Authorization  Act  for 
Fiscal  Year  1993,  contracts  that 
require  the  use  of  a  Class  I  ODS 
cannot  be  awarded  unless 
approved  at  a  level  no  lower 
than  a  general  or  a  flag  officer 
or  Senior  Executive  Service 
member . 

Therefore,  one  of  the  most 
important  features  of  TSES  is 
the  ODS  information  contained  in 
three  sections  of  the  View  Data 
Menu.  The  first  (ODS)  lists  ODS 
names  and  their  corresponding 
chemical  names;  the  next  section 
(0DS2 )  includes  the 
NAVSEA/NAVSUP/NAVAIR/MARINE 
CORPS /MILITARY  SEALIFT  COMMAND 
AUTHORIZED  USERS  LIST  FOR  the 
ODS  RESERVE  and  the  respective 


memos  authorizing  these  users, 
and  the  third  section  (OPAs) 
contains  ODS  Procurement 
Approvals .  The  complete 
NAVSUPSYSCOM  msg  211841Z  Dec  93 
is  also  on-line,  providing 
guidance  for  procurement  and 
turn-in  of  Navy  ODS  HALON 
requirements.  As  stated  in  this 
reference.  Fleet  Industrial 
Supply  Centers  are  NOT  allowed 
to  deviate  from  TSES'  table  of 
authorized  users. 

Batch  File:  The  system  screens 
the  ASCII  text  file  of  stock- 
numbered  requisitions  and  batch 
processes  them  through  TSES. 
After  processing,  it  separates 
for  review  the  questionable 
requisitions,  saves  them  in  the 
requisition  history  file  and 
creates  and  saves  the  summary  of 
TSES  actions  on-line  to  view. 

Historical  Files:  TSES 
maintains  a  file  of  requisitions 
processed  and  their  summaries, 
which  are  automatically  purged 
after  six  months. 

Help:  (FI  Key),  TSES 

incorporates  an  extensive, 
context-sensitive  Help  system, 
plus  an  On-Line  Tutorial  and 
User's  Manual. 

Desk  Top  Manager:  TSES  has  its 
own  Calculator,  Appointment 
Book,  and  Calendar. 

Report  Generator:  As  a 

management  tool  or  as  a  means 
for  each  user  to  organize  his 
data,  a  Report  Generator  is 
integrated  into  the  TSES 
software  package. 

THE  PLAYERS 

NAVSUP  is  responsible  for 
program  oversight,  implementa¬ 
tion  policy,  guidance,  and 
demonstration  and  training  at 
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user  activities;  NSPCC,  as 
NAVSUP's  agent,  provides 
programming,  database 
administration  and  technical 
support  in  the  form  of  data 
collection,  demonstration  and 
training  at  user  activities, 
item  verification  and  input,  and 
integration  and  redistribution 
of  database  updates  and  program 
modifications . 

Technical  and  procurement 
personnel  at  user  activities  are 
responsible  for  processing 
requisitions  via  TSES  logic 
guide  in  compliance  with  CNO's 
HMC&M  Program. 

TSES  -  IN  THE  REALM  OF  CNOs 
HMC&M  PROGRAM 

As  indicated  previously  in 
the  ABSTRACT  of  this  paper, 
HAZARDOUS  MATERIAL 
MANAGEMENT/ ENVIRONMENTAL  ISSUES 
are  identified  in  the  Navy 
Supply  Corps  Newsletter  of 
September/October  1993  as 
PREEMINENT  ISSUES.  TSES  is  at 
the  forefront  of  these  issues 
and  is  supporting  policy  through 
the  implementation  of  OPNAVINST 
4110.2,  Hazardous  Material 
Control  and  Management  (HMC&M) 
of  01  May  90,  which  establishes 
guidance  and  requirements  for 
the  life- cycle  control  and  total 
quality  management  (TQM)  of 
hazardous  material  (HM)  acquired 
and  used  by  the  Navy.  As  the 
TSES  user  base  grows,  the 
instructions/ regulations 
necessary  to  comply  with  the 
HMC&M  Program  will  permeate  the 
Naval  community  in  one  self- 
contained  software  package.  Some 
examples  of  how  TSES  implements 
OPNAVINST  4110.2  follow: 

Paragraph  5(a)  identifies  "HMC&M 
as  a  life- cycle  material  and 
equipment  requirement  involving 
all  elements  of  the  Navy.  The 
program  requires  HMC&M  actions 


from  concept  formulation  of  a 
new  or  modified  Navy  system 
through  research,  development, 
acquisition,  production, 
operation,  and  the  final 
disposition  phases.  It  involves 
line  management  at  all  levels 
and  action  of  all  concerned  with 
each  phase  of  acquisition  of 
systems,  components,  material, 
or  parts."  TSES  simplifies  this 
process  by  providing  a  PC-based 
software  system  designed  to 
provide  advisory  support  to 
technical  and  procurement 
personnel  via  a  step-by-step 
logic  guide .  It  further 
facilitates  line  management  at 
all  levels  by  providing  a 
Technical  Screen  Summary,  which 
is  attached  to  each  requisition 
as  it  flows  through  the  system 
and  reiterates  all  TSES  actions 
completed  and  necessary  for  each 
acquisition. 

Paragraph  5 (c)  states  that  the 
"Occupational  Safety  and  Health 
Act  (OSHA)  established 
requirements  for  training  and 
informing  workers  of  the  hazards 
from  the  chemicals  in  their 
workplaces."  TSES  incorporates 
the  Ships  Hazardous  Material 
List  (SHML)  as  an  on-line 
database.  The  SHML  not  only 
identifies  hazardous  items  which 
are  authorized  or  prohibited 
onboard  ship,  but  contains  an 
Ozone  Depleting  Substance  (ODS) 
indicator  field  identifying 
those  items  which  contain  ODS. 

Paragraph  6 (b)  asserts  that 
"Navy  activities  shall  comply 
with  all  Department  of  Defense 
(DOD)  standards,  directives, 
instructions,  and  regulations 
related  to  Hazardous  Material 
(HM)  and  Hazardous  Waste  (HW) . " 
TSES  incorporates  FAR  CLAUSE 
52.223-3  Material  Safety  Data 
Sheet  (MSDS)  ,  DFARS  CLAUSE 
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252.223-7001  Hazard  Warning 
Label  Exceptions,  ODS 
requirements,  including 
technical  certification  prior  to 
purchase  and  states  that  ODS 
items  are  reclaimable/recyclable 
and  must  not  be  sent  to  the 
Defense  Reutilization  Management 
Office  (DRMO)  or  sold  to  a 
NON-DOD  entity .  TSES  also 
provides  the  SHML  Feedback 
Report  (SFR)  and  NAVSUP  Form  87 
requirements.  All  of  these 
regulations  and  more  are  also 
identified  in  the  Technical 
Screen  Summary  portion  of  TSES 
and  must  accompany  the 
requisition  to  enlighten  all 
acquisition  process  participants 
in  the  essential  environmental 
requirements . 

Paragraphs  6(c)  (1) - (5)  state 

that  the  "Navy  shall  control  and 
reduce  the  amount  of  HM  used  and 
generated  by  upfront  HM  control 
in  acquisition,  procurement, 
supply,  and  utilization  through 
the  development  of : 

acceptable  local  mechanisms 
at  shore  activities  to  identify 
materials  in  the  system  that  are 
hazardous. . . .activity  Authorized 
Use  Lists  (AUL) ... .a  plan  to 
review  Navy  specifications  that 
direct  use  of  HM  or  HW  to 
determine  if  any  changes  are 
needed  to  further  minimize  the 
use  of  HM  and  the  generation  of 
HW. . . .mechanisms  for 
substituting  less  hazardous 
material  for  HM  where 
technically  feasible. .. .an 
overall  HMC&M  program 
encompassing  Hazardous 
Communication  (HAZCOM)  to 
promote  and  protect  the  health 
and  safety  of  Navy  workers, 
systems,  system  components,  and 
the  environment. ..." 

TSES  aids  the  requisition 


processor  in  the  implementation 
of  these  policies  and  in  control 
of  HM  throughout  the  acquisition 
process  by  providing : 

the  first  standardized 
database  for  shore  activities  to 
identify  hazardous  materials  in 
the  system  by  assimilating  the 
local  AUL  into  the  TSES 
format. . . . the  creation  of 
activity  AULs. . .  .review  of  Navy 
NSNs  and  specifications 
necessary  to  implement  CNOs 
HMC&M  Program  through  the 
continual  update  of  six 
databases  which  reside  in  TSES, 
and  the  specifications  which 
impact  them. . . .replacement  NSNs 
in  the  SHML  and  local  AUL  where 
technically  feasible  and 
authorized  plastic  alternatives 
in  the  PRIME  database . .. .an 
overall  HMC&M  software  program 
encompassing  communication/ 
dissemination  of  hazardous 
regulations /requirements  through 
automatic  "pop-up"  messages  and 
a  Technical  Screen  Summary. . . . 

Paragraph  8(b) (3)  tasks  the 
Chief  of  Naval  Research  (CNR) 
with  the  responsibility  of 
issuing  and  ensuring  compliance 
'with  codes,  standards,  and 
regulations  regarding 
HMC&M. . . . TSES  creates  an 
accessible  tool  to  disseminate 
this  information  to  its  users 
from  ship  to  shore ... .as 
regulations  change,  TSES  will 
continually  be  updated. 

Paragraph  8(c)(3)  tasks  the 
Commander,  NAVSUP,  to  ensure  the 
accurate  assignment  and  use  of 
NSNs  for  Navy  HM. . . . TSES 
requires  the  completion  of  an 
SFR  report  for  all  non-standard 
hazardous  items  to  enable  the 
assignment  of  NSNs  for  afloat 
acquisitions  and  a  NAVSUP  Form 
87,  Request  for  NSN  assignment, 
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for  non-standard  hazardous 
ashore  acquisitions . 

Other  instructions  which 
impact  TSES  and  its  program  are: 
DODINST  6050.5  DOD  Hazard 
Communication  Program  of  29  Oct 
90,  DODDIR  4210.15  Hazardous 
Material  Pollution  Prevention  of 
27  July  89,  OPNAVINST  5100. 19C 
Navy  Occupational  Safety  and 
Health  Program  Manual  for  Forces 
Afloat  of  19  Jan  94,  OPNAVINST 
5100. 23C  Navy  Occupational 
Safety  and  Health  (NAVOSH) 
Program  Manual  of  13  Aug  93  and 
OPNAVINST  5090. IB  Environmental 
and  Natural  Resources  Program 
Manual  of  01  Nov  94. 

MAINTENANCE  AND  VERIFICATION 

NSPCC  is  responsible  for 
verification  of  all  information 
in  TSES  and  the  maintenance  of 
TSES  data.  As  inputs  are 
received  from  TSES  users,  they 
are  researched  through  FEDLOG, 
Hazardous  Material  Information 
System  (HMIS) ,  Defense  Logistics 
Services  Center  (DLSC) ,  etc. 
prior  to  addition  to  the 
Non-Standard  database. 

Quarterly,  TSES  lists  and 
reference  tools  are  updated  as 
necessary;  as  well  as,  the  Non- 
Standard,  SHML,  AMAL/ADAL  and 
PRIME  databases.  The  Local  and 
Local  AUL  are  updated  by  each 
user  activity. 

Each  quarter,  NSPCC 
distributes  an  updated  software 
package  to  all  users,  including 
a  quarterly  newsletter,  entitled 
TSES  TIDBITS,  which  contains 
informative  information 
pertinent  to  the  TSES  Program 
and  its  users . 

SUMMARY 

TSES  is  not  just  another 
stand  alone  application.  It 
implements  the  CNOs  HMC&M 


policies  in  an  easy-to-use 
software  system  which  can  be 
accessed  worldwide.  It  also 
creates  a  desktop  manager 
allowing  the  user  to  access 
eight  external  programs,  as  well 
as  providing  the  user  a 
calculator,  appointment 
calendar,  report  generator,  etc. 
We  strive  to  make  TSES  the  most 
knowledgeable  expert  system 
available.  Most  of  the 
information  in  TSES  has  been 
added  from  trips  to  user 
activities  because  we  want  TSES 
to  work  for  the  people  who  use 
it . 

The  TSES  program  has  grown 
considerably  since  its 
inception.  We  have  installed 
TSES  at  75  shore-based 
activities,  including  all  Fleet 
and  Industrial  Supply  Centers 
(FISCs)  and  on  20  ships. 

Recently,  the  Aviation  Supply 
Office  (ASO)  evaluated  and  plans 
to  implement  TSES  in  conjunction 
with  their  part  number 
database/requisition  processing 
system.  As  stated  previously, 
TSES  has  sufficient  procurement 
and  technical  data  fields  to 
accommodate  the  ICP  database  and 
to  be  the  vehicle  that  the  fleet 
and  shore  activities  can  use  to 
interactively  obtain  and  provide 
technical  data  during  the 
technical  screening  process.  We 
are  providing  the  POLICY,  TOOLS, 
and  TRAINING  which  position  TSES 
to  be  the  provider  of  choice  for 
the  dissemination  of  environ¬ 
mental  information  to  fleet 
units  and  shore  stations. 
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THE  ACQUISITION  DESKBOOK 
“A  NEW  WAY  OF  DOING  BUSINESS” 
Frances  M.  Valore 

Defense  Systems  Management  College 


ABSTRACT 

The  Acquisition  Deskbook  concept 
embraces  the  notion  that  it  is  good 
business  practice  to  let  the  market 
processes  create  and  nurture  the  “Best  of 
Breed.” 

The  Under  Secretary  of  Defense  (A&T) 
chartered  the  Automated  Acquisition 
Information  Process  Action  Team  (AAI 
PAT)  to  define  the  vision  and  build  the 
roadmap  for  implementation  of  a  process 
to  disseminate  information  throughout 
DoD.  The  vision  and  the  roadmap 
begins  with  the  Acquisition  Deskbook, 
the  Deputy  Under  Secretary  of  Defense 
(AR)  initiative,  and  expand  this  concept, 
to  institutionalize  a  “new  way  of  doing 
business  within  DoD.” 

The  Acquisition  Deskbook 

The  Deskbook  will  be  an  enabling 
technology  to  promote  new  ways  of 
thinking,  and  new  ways  to  conduct 
business.  The  Deskbook  concept  changes 
not  only  the  means  by  which  acquisition 
professionals  are  provided  reference 
information  to  make  informed  decisions, 
but  will  change  the  way  we  view  business 
decision  options.  The  Deskbook 
nurtures  these  changes  by  providing 
acquisition  professionals  with  current  and 
complete  information  at  their  fingertips, 
presented  in  a  configuration 
corresponding  to  thinking  processes. 


Three  integral  components  make  up  the 
Deskbook:  A  Reference  Set,  a  Tool 
Catalog,  and  an  electronic  Acquisition 
Management  Forum  (AM  Forum).  A 
Joint  Functional  Team  (JFT),  consisting 
of  representatives  from  each 
Component  Acquisition  Executive 
(CAE),  the  Office  of  the  Secretary  of 
Defense  (OSD)  and  the  Defense  Systems 
Management  College  (DSMC)  will 
identify  the  data  to  be  included  in  the 
Reference  Set,  a  structured  collection  of 
categorized  and  certified  acquisition 
information  vital  to  acquisition 
management.  The  Tool  Catalog  is  a 
database  of  software  acquisition  tools, 
currently  available  or  under  development, 
describing  the  functional  capability, 
systems  compatibility,  and  a  point  of 
contact  for  each  individual  tool.  The 
AM  Forum  is  an  unstructured  and 
informal  electronic  information  exchange 
to  facilitate  the  rapid  flow  of  good  ideas 
and  information  throughout  the 

acquisition  community. 

Acquisition  professionals  will  have  an 
electronic  “one  stop  shopping”  center 
that  provides  key  reference  information. 
To  make  the  service  widely  available,  the 
Deskbook  will  be  an  integrated  software 
application  that  can  be  used  as  a  stand¬ 
alone  installation,  on  a  network,  or 
accessed  remotely  via  the  Internet, 
World-Wide-Web,  or  dial-up  modem. 


602 


BACKGROUND 

“With  successful  implementation, 
acquisition  reform  should  change  the 
way  we  conduct  business  within  the 
DoD.”  -  Honorable  Paul  J.  Kaminski, 
The  Under  Secretary  of  Defense 
(A&T). 

In  order  to  implement  this  change  in  the 
way  we  do  business,  Dr  Kaminski 
chartered  the  Automated  Acquisition 
Information  Process  Action  Team  (AAI 
PAT)  to  “...Define  a  vision  and  build  a 
roadmap  to  institutionalize  an 
automated  acquisition  information 
process  to  provide  current  and 
comprehensive  information  and  tools 
for  the  Office  of  the  Secretary  of 
Defense  (OSD),  Services  and  Program 
Managers  to  effectively  and  efficiently 
buy  weapon  systems.” 

To  realize  this  vision,  the  PAT  developed 
four  goals  to  describe  the  desired  end- 
state  acquisition  information 
environment: 

Goal  1:  An  automated  acquisition 

information  process  will  exist  that 
provides  timely  and  effective 
information  sharing. 

Goal  2:  A  streamlined  automated 

tracking,  monitoring  and  reporting 
information  process,  integrated  with 
program  management  planning  and 
execution  tools,  will  be  in  place  and 
operating. 

Goal  3:  A  “library”  of  automated 
acquisition  tools  and  information  will 
exist  and  be  accessible  to  all. 


Goal  4:  Training  and  support  on  AAI 
systems  will  be  fully  institutionalized. 

The  PAT  plan  of  action  to  define  the 
vision  and  develop  the  roadmap 
coalesced  into  two  systems  or  processes: 
the  creation  of  an  Acquisition  Deskbook 
system,  and  the  development  of  an 
Automated  Program  Status  Reporting 
(APSR)  system.  They  are  described 
below. 

The  Acquisition  Deskbook 

The  Deskbook  will  be  an  integrated, 
electronic  desktop  system  for  the 
exchange  of  information  to  support  the 
acquisition  community.  Timely  access  to 
complete,  relevant  information, 
structured  to  advance  managerial 
discretion,  will  promote  a  new  way  of 
doing  business. 

Three  separate,  integrated  components 
address  unique  elements  of  acquisition 
information.  The  Reference  Set  is  a 
structured  information  set  -  collected, 
identified,  categorized,  and  certified  by  a 
Joint  Functional  Team  (JFT).  This  Team 
consists  of  representatives  from  each 
Component  Acquisition  Executive 
(CAE),  the  Office  of  the  Secretary  of 
Defense  (OSD)  and  the  Defense  Systems 
Management  College  (DSMC  ).  The 
Tool  Catalog  will  be  a  database  of 
software  acquisition  tools  currently 
available  or  under  development,  and  will 
describe  the  functional  capability, 
systems  compatibility,  and  a  point  of 
contact  for  each  individual  tool.  The 
Aquisition  Management  Forum  will  be 
an  unstructured,  informal,  real-time 
information  exchange  to  facilitate  idea 
generation  and  information  exchange 
throughout  the  acquisition  community. 
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The  Deskbook  initial  phase  will  be 
operational  in  September  1995,  given 
direction  and  timely,  sufficient  funding. 

The  Deskbook  will  be  an  electronic  “one 
stop  shopping”  center  that  provides  the 
acquisition  community  with  key 
reference  information.  It  will  be  an 
integrated  software  application  that  can 
be  used  as  a  stand-alone  installation,  on  a 
network,  or  accessed  remotely  via  the 
Internet  or  dial-up  modem. 

The  Automated  Program  Status 
Reporting  System  (APSR) 

Goal  #2  in  Dr  Kaminski’s  vision  was 
addressed  external  to  the  initial 
Deskbook  system;  however,  the  plan  has 
the  reporting  process,  over  time, 
integrated  with  program  management 
planning  and  execution  tools.  The 
overall  acquisition  information  process 


takes  advantage  of  emerging  technology, 
to  further  an  evolving  acquisition  culture 
of  good  business  practices. 

The  APSR  system  originates  with  the 
development  of  a  data  dictionary  that 
defines  selected  elements  of  program 
information  needed  for  program 
oversight.  This  data  dictionary  will  be 
the  foundation  for  a  database  developed 
to  support  an  automated,  non-intrusive 
monitoring,  tracking,  and  reporting 
process.  This  system  will  replace  other 
forms  of  oversight  reporting,  and  provide 
timely  and  accurate  information,  while 
reducing  costs  associated  with  duplicated 
program  management  efforts. 

The  Army  will  establish  and  manage  the 
development  of  the  APSR  system,  and 
will  coordinate  appropriate  APSR 
actions  with  the  System  Acquisition 
Management  Corporate  Information 
Management  (SAM-CIM)  efforts  to 
achieve  consistency  and  efficiencies. 

The  Director,  DUSD(A&T)  API,  will 
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chair  the  JFT  governing  the  Deskbook 
and  the  APSR  system  efforts,  and  the 
SAM-CIM  efforts. 

Numerous  official  and  unofficial  studies 
and  reports  have  documented  the  need 
for  acquisition  managers  to  have  valid, 
accurate,  and  timely  information  to  avoid 
“surprises,”  often  in  the  form  of 
unexpected  project  cost  increases  and 
schedule  slippages.  Historically,  the 
bureaucratic  response  has  been  to 
develop  a  new  “rule”  to  prevent  that 
specific  re-occurrence.  Unfortunately, 
these  rules  are  perceived  as  inviolate  and 
inflexible,  and  as  such,  become  a 
substitute  for  thoughtful  judgment  and 
decisive  action. 

Managerial  expertise,  and  discretional 
judgment  has  always  been,  and  continues 
to  be,  critical  to  successful  results.  In 
the  politically  charged  defense  arena  this 
is  especially  true.  No  amount  of 
organized,  systematic  ‘checklists’  can 
substitute  for  the  valuable  art  of 
decision-making  based  on  cumulative 
management  expertise,  thoughtful 
analysis  of  a  situation,  and  finally,  the 
courage  to  make  a  judgment  call  in  a 
risk-filled  environment. 

The  current  environment,  within  which 
REGO  (REinventing  Government)  is 
encouraged  at  the  highest  levels,  brings 
opportunities  to  re-insert  a  heightened 
awareness  of  the  managerial  discretion 
available  to  government  careerists. 
Information  flow  processes  should  be 
structured  in  ways  that  promote  this 
awareness  to  enable  informed  decisions 
by  acquisition  professionals.  The 
organization  of  Deskbook  information 
enables  users  to  more  easily  distinguish 


between  that  which  is  Public  Law  or 
Directive  and  which  is  Discretionary. 

Technology  exists  today  to  place  current 
policy,  guidance  and  automated  tools  for 
program  management  at  the  fingertips  of 
every  acquisition  professional.  More 
timely  and  accurate  program  information 
can  be  made  available  to  decision  makers 
and  their  staffs. 

The  Reinventing  Government 
philosophy,  coupled  with  available  and 
emerging  technology,  offers  chances  to 
make  real  and  enduring  differences.  The 
political  and  economic  climate 
encourages,  if  not  forces,  creative 
solutions  while  the  Department  braves 
significant  redefinitions  of  its  roles. 

The  Deskbook  concept  uses  available 
technology  to  encourage  new  ways  of 
managing  acquisition  business.  It  offers 
opportunities  to  reduce  workload  on 
program  managers,  to  eliminate  the 
requirement  for  compiling  periodic 
reports,  to  greatly  reduce  the  cost  of 
publishing  acquisition  directives,  and  to 
facilitate  the  flow  of  useful  information 
throughout  the  acquisition  workforce. 

CHANGING  A  CULTURE 

...  Envision  a  near  future  that 
endorses  application  of  emerging 
technology,  creative  thinking,  and 
rewards  proactive  decision  making ... 

Problem  1:  Dissemination  of 

Acquisition  policy  and  approved 
practices  can  take  weeks  or  even  months 
to  reach  the  members  of  the  acquisition 
workforce.  Also,  no  fast  path  exists  to 
share  “good  ideas”  across  the 
Components. 
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Goal  1:  An  automated  acquisition 
information  process  will  exist  that 
provides  timely  and  effective  sharing  of 
information. 

Problem  2:  Acquisition  managers  spend 
significant  time  and  resources  to 
generate  oversight  reports.  Responsible 
decision  makers  do  not  always  have 
access  to  the  most  current  information. 

Goal  2:  A  streamlined  automated 
tracking,  monitoring  and  reporting 
information  process,  which  integrates 
with  program  management  planning  and 
execution  tools,  will  be  in  place  and 
operating. 


The  Deskbook  system  addresses  the 
problems  and  goals  #1,  3,  &  4.  The 
APSR  System  addresses  Goal  #2. 

The  Deskbook  system  will  have  a  user- 
friendly  graphical  display  with  an 
intuitive  user  interface  to  minimize 
training  requirements.  A  flexible  user 
interface,  navigational  mechanisms  and 
search  functions  will  satisfy  new  users  as 
well  as  experts.  Information  will  be 
presented  to  the  user  through  standard 
graphical  user  interfaces  (i.e.,  Windows 
and  Macintosh).  The  system  is  designed 
to  be  Internet-compatible  and  installable 
on  either  a  Local  Area  Network  or  a 
standalone  computer. 


Problem  3:  There  is  no  centralized 
list  of  acquisition  management  tools.  No 
procedure  exists  by  which  acquisition 
managers  can  investigate  the  existence 
of  a  software  tool  to  meet  their  needs 
before  developing  their  own.  As  a 
result,  acquisition  managers  devote  a 
significant  amount  of  time  and  money 
each  year  developing  automated  tools 
that  already  exist. 

Goal  3:  A  “library”  (e.g.,  inventory, 

index,  catalog)  of  automated  acquisition 
tools  and  information  will  exist  and  be 
accessible  to  all. 


The  library  of  information  will  be 
developed  logically  through  an  iterative 
approach  and  will  be  implemented  using 
a  continuously  evolving  database.  The 
Deskbook  will  link  to  existing  databases 
and  services,  where  appropriate,  to  avoid 
costly  duplication. 

The  Reference  Set  volume  of  the 
Acquisition  Deskbook  provides  the 
opportunity  to  support  a  basic  change  in 
the  acquisition  culture.  Information  in 
the  Reference  Set  is  organized  into 
categories: 


Problem  4:  Acquisition  workforce 

training  does  not  adequately  use  the 
many  tools  available  in  the  normal 
curriculum.  Existing  tools  are  not 
widely  publicized  to  the  acquisition 
workforce. 

Goal  4:  Training  and  support  on  AAI 
systems  will  be  fully  institutionalized. 


Guiding  Principles  Directive 

Information 


Institutionalized 

Knowledge 

Sage  Information 


Discretionary 

Information 

Expert  wisdom, 
Lessons  Learned 
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By  organizing  the  Reference  Set 
information  into  categories,  the 
Deskbook  sends  a  clear  message  that  the 
use  of  discretion  and  judgment  are 
mainstream  elements  of  our  business 
process.  The  Reference  Set  information 
categories  require  multiple  levels  of 
certification.  Guiding  principles  must  be 
approved  by  an  appropriate  authority. 
Institutionalized  knowledge  will  be 
approved  by  the  process  owners,  and 
Sage  Information  will  come  from  anyone 
with  applicable  experience  and  a 
willingness  to  share  ideas. 

The  information  is  further  divided  to 
focus  on  specific  interests  and 
audiences,  and  to  differentiate  between 
DoD  and  Component  applications. 
Users  will  have  a  variety  of  ways  to 
access  the  information,  and  must  re¬ 
orient  their  search  for  the  right  “rule,” 
and  become  familiar  with  new 
information  categories. 

Guiding  Principles.  This  information 
describes  the  products  and  processes  the 
acquisition  community  must  produce  and 
follow.  This  area  includes  mandatory 
provisions,  directives  policy,  operating 
procedures  and  regulations.  Information 
in  this  category  must  be  approved  by  the 
appropriate  Acquisition  Executive  (AE) 
or  designee.  Example:  DoD  5000 
Series. 

Institutionalized  Knowledge.  This 
information  describes  the  products  and 
processes  the  acquisition  community 
may  produce  and  follow.  This  area 
includes  alternative  approaches  that  have 
been  tried  and  approved  for  use,  either 
generally  or  for  specific  applications. 
Information  in  this  category  must  be 
approved  by  the  process  owner 


designated  by  the  appropriate  AE. 
Example:  Service-specific  requirements. 

Sage  Information.  This  information 
describes  learned  advice  from  functional 
experts,  lessons  learned  from  past 
experiences,  status  of  “pilot”  programs, 
and  general  information  that  is  neither 
directive  nor  discretionary.  This 
information  must  be  approved  by  officials 
who  have  been  delegated  this  authority 
by  an  AE.  Example:  National 

Performance  Review  Pilot  Programs. 

Information  in  the  Reference  Set  is 
further  sub-divided  into  Classes: 

-  DoD-Wide 

-  Service/Agency-Wide 

-  Buying-Activity  Specific 

The  Defense  Acquisition  Executive 
(DAE)  controls  the  approval  process  for 
the  DoD-Wide  class  of  information. 

The  Service  Acquisition  Executive 
(SAE)  or  Agency  senior  acquisition 
official  controls  the  approval  process  for 
The  Service/ Agency-Wide  class  of 
information. 

The  Buying-Activity  senior  acquisition 
official  controls  the  approval  process  for 
The  Buying-Activity  Specific  class  of 
information. 

The  USD  (A&T)  will  establish  a  process 
to  certify  and  categorize  the  information 
to  be  included  in  the  Deskbook  in  the 
“DoD-Wide”  class  of  information.  This 
process  will  be  developed  in  conjunction 
with  the  current  rewrite  of  the  DoD  5000 
principle  acquisition  documents. 
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Each  Service  and  Agency  will  establish  a 
process  to  categorize  and  certify  the 
information  to  be  included  in  the 
Deskbook  in  the  specific 
“Service/Agency-Wide”  class  of 
information,  e.g.,  Air  Force,  Navy, 
Army,  DLA  and  SOCOM. 

Each  Buying  Activity  will  establish  a 
process  to  categorize  and  certify  the 
information  to  be  included  in  the 
Deskbook  in  the  specific  “Buying 
Activity”  class  of  information,  e.g.,  ESC, 
NAVAIR  and  TACOM. 

Envision  the  Deskbook  as  the  living  tree 
trunk,  that  encompasses  the  combined 
knowledge  of  the  acquisition  community. 
The  Services  and  Agencies  feed  practical 
experience  information  up  through  the 
roots,  and  OSD  and  the  Administration 
feed  guidance  and  fundamental  principles 
down  through  the  branches.  In  the 
center,  the  information  comes  together  in 
the  Reference  Set. 

The  appropriate  level  of  certification  is 
important  to  ensure  the  process  is 
streamlined  and  disseminated  rapidly 
throughout  DoD.  Guiding  Principles 
information  should  change  infrequently. 
This  information  has  the  greatest  effect 
on  the  practice  of  acquisition,  and  the 
certification  process  should  have  high 
level  review  and  coordination.  The 
relevant  AE  is  the  appropriate 
certification  official. 

The  Institutional  Knowledge  category 
provides  approved  alternative  practices. 
Certification  should  be  delegated  to 
appropriate  process  owners  within  the 
Component  or  Department. 


The  Sage  Information  category  collects 
and  promulgates  promising  ideas,  lessons 
learned  and  advice  throughout  the 
Department.  The  certification  process 
should  be  delegated  to  the  lowest 
possible  level.  A  lengthy  and  top-heavy 
review  process  for  the  Sage  Information 
category  would  be  counter-productive. 

Each  Component  currently  has  personnel 
assigned  to  develop  and  coordinate 
policy  information;  these  individuals’ 
efforts  should  be  realigned  to  be 
consistent  with  the  Deskbook  concept 
and  the  revised  information  categories. 
The  re-write  of  the  DoD  5000  series 
documents  will  formalize  these 
information  categories. 

A  Joint  Program  Office  (JPO)  will 
manage  the  overall  Deskbook  System. 
The  existing  Air  Force  Acquisition 
Model  (AFAM)  program  office  will  form 
the  initial  cadre  for  the  Deskbook  effort 
to  take  advantage  of  their  experience  in 
building  an  acquisition  reference  system 
for  the  Air  Force,  and  their  existing 
infrastructure.  The  program  manager 
will  rotate  through  the  Services  every  3 
years  in  the  following  sequence:  Air 
Force,  Army,  Navy/Marine  Corps.  The 
AFAM  organization  will  become  the 
Deskbook  Program  Office. 

In  addition  to  the  overall  Deskbook 
management  functions,  the  Air  Force 
AFAM  Program  Manager  will  be 
responsible  for  the  establishment  and 
management  the  Reference  Set 
component  of  the  Deskbook. 

The  Joint  Functional  Team  (JFT) 
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These  information  collection  and 
certification  processes  offer  opportunities 
for  a  “Joint”  look  across  all  Services  and 
Agencies,  to  review  other  components’ 
Guiding  Principles  and  Institutional 
Knowledge  information  for  potential 
application  across  all  the  Components. 
The  Reference  Set  Information 
Coordinators  from  each  of  the 
Component  review  boards  will  form  a 
JFT.  This  Team  will  coordinate 
potential  joint  guiding  principles  and 
institutional  knowledge  and  present  their 
recommendation  to  a  Joint  Service 
Review  Board  for  endorsement  across 
the  Components. 

The  JFT  will  work  with  the  Deskbook 
Joint  Program  Office  (JPO)  to  establish 
the  “views”  into  the  information.  The 
Deskbook  will  contain  a  tremendous 
quantity  of  data;  therefore,  the  available 
methods  for  the  acquisition  workforce  to 
view  that  information  are  critical  to  the 
Deskbook  becoming  the  reference  source 
the  workforce  chooses  to  use. 

The  JFT  will  approve  requirements  and 
perform  oversight  functions  for  the  DoD 
Acquisition  Deskbook.  The  JFT 
management  structure  ensures  a  DoD- 
wide  vision  for  automated  acquisition 
information,  while  sufficiently 
recognizing  the  importance  of 
maintaining  Service/Agency-unique 
requirements.  Each  AE  will  be 
responsible  for  establishing  internal 
processes  to  identify  and  approve 
information  for  incorporation  into  the 
Deskbook. 

To  ensure  the  Deskbook  maintains  a 
DoD  perspective,  the  JFT  will  be 
chartered  to  recommend  DAE  approval 
of  alternative  practices  and  "good  ideas" 


that  cross  Component  lines.  They  will  be 
explicitly  tasked  to  seek  out  those 
practices  and  policies  introduced  at  the 
Component  level  that  may  be  appropriate 
for  Joint  implementation.  Members  will 
represent  their  respective  Components 
on  the  JFT,  and  serve  as  the  Component 
advocate  for  the  Deskbook.. 

The  JFT  will  consist  of  senior-level 
representatives  (0-6  or  GM-15)  assigned 
from  each  of  the  following  organizations: 

•  Director,  API,  Chairperson 

•  Deputy  USD( Acquisition  Reform) 

•  Army 

•  Navy 

•  Air  Force 

•  Marine  Corps 

•  Defense  Logistics  Agency 

•  Special  Operations  Command 
•Deskbook  P.M.  (Technical  Consultant) 

•  Defense  Systems  Management  College, 

Executive  Secretariat  &  Host 


The  Tool  Catalog:  The  purpose  of 

the  Tool  Catalog  is  to  provide  a  central 
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library  or  referral  for  acquisition 
managers.  Today’s  environment  isolates 
the  multiple  markets  available  for 
determining  the  availability  of  software 
tools,  and  leads  to  widespread 
proliferation  and  duplication.  A  need 
for  a  tool  is  identified  by  a  single 
program  office  or  a  small  group  of 
offices.  They  then  develop  the  needed 
tool,  or  an  enterprising  commercial 
company  sells  them  an  existing  tool.  This 
process  is  repeated  in  isolation 
throughout  the  acquisition  community. 
Individual  acquisition  managers  do  not 
have  the  information  available  to 
determine  whether  a  tool  already  exists 
elsewhere.  The  Tool  Catalog  will  reduce 
duplicate  development  and  provide  the 
following  information: 

•  Tool  functional  classification 

•  Functional  Description 

•  Assessment  of  Capability 

•  Software  and  Hardware  Compatibility 

•  Implementation/Support  Cost 

•  Identify  Current  Users 

•  Too!  Owner  and  Acquisition  Source 

•  Contract  Vehicles,  if  Any 

•  Date  Of  Last  Upgrade 

Acquisition  tools  will  be  classified 
functionally  in  the  following  categories: 

•  Industrial/Manufacturing/Quality 

Assurance 

•  Reference 

•  Program  Management 

•  Financial  Management 

•  Contract  Management 

•  Engineering 

•  Configuration  Control 

•  Test  and  Evaluation 

•  Logistics 

•  Foreign  Military  Sales 

•  Safety 


•  Security 

•  Environmental 

•  Installation  Management 

•  Construction  Management 

•  Human  Systems  Integration 

•  Others,  When  Identified 

In  addition  to  the  cataloging  function,  a 
consulting  service  will  be  provided  to 
support  the  Tool  Catalog.  As  users 
consult  the  Tool  Catalog  service,  the 
Deskbook  will  collect  trend  information 
on  the  automated  tool  needs  of 
acquisition  managers.  This  information 
can  then  be  analyzed  and  provided  to  the 
Joint  Functional  Team  (JFT).  The 
Navair  PMA-250  will  manage  the  Tool 
Catalog. 

The  Acquisition  Management  Forum: 
The  AM  Forum  component  of  the 
Deskbook  will  be  an  electronic 
information  exchange  to  facilitate  the 
flow  of  unstructured  information  within 
the  acquisition  community.  The  Forum 
offers  timely,  informal  exchanges  of 
information,  advice,  ideas  and 
consultation  and  could  enable  teaming 
opportunities  with  others  having  similar 
projects,  questions,  concerns,  or 
initiatives.  Examples  of  items  that  may 
be  included  in  the  Forum  are: 

•  Pilot  Programs  Status 

•  Acquisition  Reform  Initiatives 

•  New  Ideas,  Alternative  Practices 

•  Distance  Learning  Applications 

•  Open  Discussion  and  Feedback 

•  Notices  -  Course  Offerings, 

Conferences,  Seminars 

•  New  or  Emerging  Policy 

The  AM  Forum  will  communicate 
survey  findings,  concerns  and  current 
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issues,  coordinate  expert  functional 
responses  to  queries,  and  transmit  real¬ 
time  status  of  pending  legislation,  PPBS 
updates  (Committee  Mark-ups,  etc.)  and 
other  items  of  interest  indicated  by 
users/customers.  An  AM  Forum  central 
focal  point  will  facilitate  the  exchange  of 
top-down,  bottom-up  and  horizontal 
information  that  is  consistent,  valid, 
accurate  and  timely.  The  Forum  will 
have  external  interface  and  coordination 
(legislation,  executive  actions,  hot  news) 
and  internal  interface  and  coordination 
(OSD,  Services  and  Agencies). 

The  AM  Forum  organizational  functions 
include  information  collecting, 

screening,  directing  and  disseminating, 
but  does  not  include  certification  or 
approval  functions.  The  key  to  holding 
a  forum  that  people  want  to  use  has  less 
to  do  with  fancy  graphics  or  quick 
connections,  but  with  the  people  and  the 
information  they  find  there.  The  intent  is 
not  to  see  warm  and  fuzzy,  sanitized, 
approved  information,  but  instead  to 
encourage  open  discourse,  a  gloves-off 
exchange  of  ideas,  and  rapid 
communication  among  multiple  DoD 
community  users.  The  AM  Forum  will 
be  available  via  Internet,  telefax  and 
telephone  message  24  hours  a  day,  and 
staffed  during  normal  duty  hours. 

DSMC  will  establish  and  manage  the 
AM  Forum  and  use  the  Forum  to  support 
DAU  in  the  planning  and  distribution  of 
publicity  and  training  to  support  the 
Deskbook  system. 


Components  of  the  DoD  Deskbook 


Tod  AMForum 


The  User 


Training/Education: 

The  Defense  Acquisition  University 
(DAU)  will  coordinate  the  training  and 
education  requirements  to  ensure  the 
Deskbook  concept  is  institutionalized. 
Target  date  for  full  integration  of  the 
Deskbook  into  the  applicable  DAU 
consortium  curriculum  is  Mar  96. 

The  DAU,  upon  determination  of  need 
for  new  courses  or  additions  to  existing 
courses,  will  coordinate  the  development 
of  all  training  and  education  functions  for 
the  Deskbook,  targeting  applicable 
courses  at  all  levels. 

The  core  competencies  for  Deskbook 
instruction  should  include  an  appropriate 
emphasis  on  the  underlying  philosophy  of 
the  Deskbook,  as  an  enabling  technology 
to  disseminate  information  on  a  “new 
way  of  doing  business.”  The  Defense 
Acquisition  Corps  Functional  Boards  will 
determine  the  need  to  develop  new 
competencies,  or  revise  their  respective 
functional  competencies  accordingly,  and 
task  the  DAU  to  develop  or  revise 
applicable  course  curricula. 

TO  NEW  WAYS  OF  THINKING 
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The  Deskbook  product  and  associated 
services  has  been  a  mission  motivated  by 
an  important  vision.  The  AAI  PAT 
members  were  driven  to  achieve  our 
goals,  because  it  mattered  to  each  of  us, 
albeit  in  a  variety  of  ways. 

Don’t  consider  the  job  finished  now  that 
the  PAT  has  completed  its  initial  mission 
-  to  define  a  vision  and  develop  a 
roadmap.  Each  of  us  on  the  “global 
team”  has  a  vested  interest  in  some  part 
or  all  these  goals  and  objectives.  The 
PAT  achieved  hard-won  alliances  on 
issues  that  mattered  most,  and  a 
workable  consensus  on  others. 

Along  the  way,  we  bought  into  the 
process  that  brought  about  the 
commitment.  We  will  take  with  us  this 
process  of  sharing  that  creates 
enthusiasm  to  enhance  our  futures.  We 
found  ways  to  successfully  bridge 
differences  and  truly  bring  the  best  of 
each  into  a  product.  The  product  is  a 
“beginning”  we  will  all  expand  to  make  a 
difference  in  our  fields. 

Although  new  ideas  are  often  resisted, 
passions  for  specific  ideas,  principles, 
people  or  things  reside  within  us  all.  In 
the  quest  to  define  the  vision,  goals  and 
objectives  for  DoD’s  automated 
acquisition  information  future,  each 
member  of  the  team  worked  to  construct 
a  process  within  which  individual 
passions  could  thrive.  The  final  vision 
will  indeed  incorporate  these  ideas,  and  is 
designed  to  seek  more  shake-up  ideas. 


globally.  The  team  and  its  dynamics  are 
representative  of  DoD’s  diversity  of 
backgrounds  and  interests.  It  proved  to 
be  an  encouraging  testing  ground  for 
paradigm  change.  Laboriously,  even 
sometimes  painfully,  we  achieved 
milestones  -  not  closure  -  on  the  first 
phase  of  the  Vision  Roadmap. 

New  ways  of  thinking  won’t  show  up  in 
the  mirror  one  morning.  True  measures 
of  success  are  difficult  to  define,  and  to 
achieve,  which  is  why  they  are  so  valued. 


The  Deskbook  itself  is  an  idea.  How 
good  the  idea  is  depends  upon  how  we 
use  it.  The  concept  is  much  more  than 
is  immediately  evident.  It  is  an  excellent 
beginning  to  push  us  to  expand  our  most 
rigid  boundaries. 


We  are  destined  to  find  new  ways  to 
conduct  our  business  defense-wide  and 
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1986  A.D.  Little  Industry  Survey  Report 

1990  Defense  Science  Board  Acquisition  Streamlining  Task  Force 
Interim  Reports 

May  1991  DoD/Tndustry  Survey  Report 

April  1994  DoD  Acquisition  Streamlining  Model 
DSMC  Research  Project 

Dec  1994  Oversight  and  Reporting  PAT  Report 

April  1995  Automated  Acquisition  Information  PAT  Report 
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AIR  FORCE  ACQUISITION  MODEL  (AFAM)  - 
THE  LIVING  DATA  BASE 

Daniel  M.  Whye,  Captain,  USAF 


ABSTRACT 

Nearly  everyone  is  apprehensive  when 
doing  something  new.  In  the  complex 
acquisition  process,  “first  exposures”  are 
particularly  challenging.  Fortunately,  Air 
Force  Acquisition  Command  (AFMC)  has 
fielded  an  acquisition  tool  to  meet  this 
challenge.  It’s  called  the  Air  Force 
Acquisition  Model  (AFAM).  The  AFAM 
is  a  personal  computer  database  that 
describes  AFMC’s  acquisition  “tasks” 
across  all  six  acquisition  phases.  It 
describes  tasks,  references,  wisdom  in  the 
form  of  best  practices,  lessons  learned,  and 
expert  advice;  tools  to  help  complete  the 
task,  and  timeline  information  for  more 
than  5,000  acquisition  tasks.  All  of  this 
plus  a  supplemental,  hyper- text  search  and 
retrieval  library  know  as  the  Air  Force 
Acquisition  Model  Supplement 
(AFAMSUP)  containing  170  key 
acquisition  directives  and  guides.  To 
ensure  all  of  the  information  in  each  phase 
is  comprehensive  and  complete,  the 
database  is  updated  and  expanded  with  each 
release.  The  model  version  2.1  released  in 
March  1995  is  the  seventh  release  from  the 
Acquisition  Model  Program  Office  located 
at  Aeronautical  System  Center  (ASC), 
Wright-Patterson  Air  Force  Base  (AFB), 
Ohio. 

INTRODUCTION 

Just  imagine  a  tool  at  your  fingertips  that  is 
a  repository  for  the  Air  Force  acquisition 
and  product  management  process.  Like  the 
Yellow  Pages,  a  tool  you  can  let  your 


fingers  do  the  walking  to  aid  you  in 
accomplishing  a  specific  task  or  tasks 
during  a  major  acquisition  milestone  phase. 
A  tool  that  helps  you  to  determine  where  to 
begin  a  project  and  to  avoid  reinventing  the 
wheel.  This  mega-tool  has  been  years  in 
the  making.  All  you  need  is  a  386 
computer  with  Windows,  and  an  inquisitive 
mind.  This  tool  is  the  AFAM,  a  computer 
data  base  that  describes  the  processes  of  Air 
Force  acquisition  and  product  management. 
It  is  an  automated  encyclopedia  depicting 
and  outlining  all  six  phases  of  the 
acquisition  process.  The  model  is  designed 
to  assist  research  &  development, 
acquisition,  and  support  personnel  in 
performing  tasks  for  major  weapon  system 
programs  and  non-major  acquisitions. 
Although  acquisition  is  in  its  name,  the 
model  takes  a  “cradle  to  grave”  approach 
in  the  life  cycle  of  the  system. 

General  Yates  --  The  Lightning  Bolt 

The  model  is  the  brainchild  of  General 
Ronald  W.  Yates,  commander  of  Air  Force 
Materiel  Command  (AFMC).  He 
recognized  that  in  an  era  of  downsizing, 
reduced  budgets,  and  reengineering  our 
processes,  the  Air  Force  acquisition  corps 
couldn’t  afford  to  keep  reinventing  the 
wheel.  General  Yates,  then  commander  of 
Air  Force  Systems  Command  (AFSC), 
wanted  an  automated  tool  to  define  the 
acquisition  and  sustainment  process  as  well 
as  a  method  to  share  best  practices  and 
lessons  learned  to  assist  the  acquisition  and 
logistic  communities.  He  took  the  initiative 
to  put  together  a  program  that  captured  our 
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best  practices.  During  the  transition  to 
AFMC,  he  and  General  Charles  C. 
McDonald,  then  commander  of  Air  Force 
Logistics  Command  (AFLC),  established 
an  integrated  program  office  in  the  Center 
for  Supportability  and  Technology  Insertion 
which  is  part  of  ASC.  The  program  office 
was  chartered  to  develop  a  model  to 
capture  the  Air  Force  acquisition  system. 

General  Yates’  emphasis  on  the  need  for 
AFAM  was  twofold.  He  said,  “We  can’t 
teach  and  we  can’t  learn.  When  we  have  to 
accomplish  a  task  that  we’ve  never  done 
before,  we  don’t  have  a  well-equipped  tool 
box  to  select  the  tool  or  blueprint  we  need 
to  accomplish  the  task.  Experience  has 
shown  we  look  for  someone  who’s  done  the 
task  before  or  we  look  for  a  recent  example 
to  use.  ” 

No  matter  what  the  scenario  is  we  may 
spend  a  lot  of  time  tracking  down  the  so 
called  “expert”  or  we  run  the  risk  of  using 
a  “bad”  example.  AFAM  captures 
successful  experiences  and  documents  and 
makes  them  available  to  everyone.  General 
Yates  said,  “During  this  period  of 
reductions,  we  are  losing  the  experienced 
people  who  have  the  corporate 
knowledge.”  “Therefore,  tools  like  the 
AFAM  will  help  us  produce  quality 
products  the  first  time.”  It  will  ease  the 
learning  curve  process  and  provide  the 
“road  map”  to  accomplish  the  job. 
Finally,  he  stated,  “It  will  allow  us  to 
quickly  convey  changes  to  the  field  —  with 
a  sense  of  urgency.” 

AFAM  Genesis 

Development  of  AFAM  began  in 
November  1991  with  a  prototype  developed 
for  demonstration  to  the  joint  staff  of  the 
Air  Force  Logistic  and  Systems  Commands 


in  January  1992.  The  AFAM  Program 
received  100  percent  endorsement  and 
program  go-ahead.  People  within  the 
command  were  tasked  to  develop  computer 
displays,  data  management  programs,  and 
task  field  organizations  to  proof  the 
embedded  processes  and  contribute  their 
wisdom  and  expertise.  Thus  the  Air  Force 
Acquisition  Model  was  bom. 

Colonel  Mike  Ferrell,  director  of 
Acquisition  Model  Program  Office,  said, 
“Our  goals  were  to  create  a  just-in-time 
training  tool  that  would  teach,  capture  best 
practices,  and  give  program  directors  the 
information  they  need  to  plan,  manage,  and 
execute  their  programs.”  We  started  with 
approximately  30  hand-picked  people  who 
represented  the  major  functional  areas,  such 
as  contracting,  financial  management,  and 
the  like,  and  we  developed  a  preliminary 
model.  “Then  we  created  a  customer 
requirements  review  board  of  people  from 
each  center  within  AFMC  and  asked  them 
to  tell  us  what  they  liked  about  our 
preliminary  model,  and  what  they  wanted 
and  needed.  They  were  the  ones  who 
shaped  the  first  model  release.  ” 

Eight  months  later,  what  emerged  was  a 
personal  computer  data  base  that 
summarized  the  latest  set  of  Air  Force 
rules,  regulations  and  policies  governing 
the  acquisition  and  sustainment  process. 
The  model  was  on  its  way  to  become  the 
“how-to”  recipe  book  using  the  best  ideas 
and  guidance  available  from  DOD  talent 
and  experience.  Subsequent  releases  occur 
every  six  months  to  ensure  the  model 
reflects  current  business  practices  and  the 
most  recent  corporate  knowledge. 
Originally  targeted  toward  the 
inexperienced,  AFAM  has  expanded  its 
audience  to  include  all  acquisition 
personnel. 
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AFAM  Matures 

AFAM  has  been  extremely  successful 
because  it  is  very  easy  to  use  and  has  high 
quality  information.  Anyone  who  can  point 
and  click  a  mouse  or  use  a  keyboard  can 
quickly  access  valuable  acquisition 
information.  The  information  in  the  model 
is  reliable  since  it’s  been  provided  by 
AFMC  experts  and  then  validated  by 
AFMC  functional  directors.  Since  it  has 
been  fielded,  users  claim  it  is  a  significant 
time-saver  and  reference  tool.  The  goal  is 
to  expand  the  AFAM  capability  from 
describing  what  is  required  to  accomplish 
basic  tasks  in  each  phase  of  the  acquisition 
life  cycle  to  actually  assisting  the  user  in 
accomplishing  tasks. 

Infrastructure 

Since  its  inception  and  with  each  AFAM 
version  release,  the  model  continues  to  add 
more  enhancements  and  data  to  the  growing 
customer  base  living  up  to  its  name  as  a 
“living  data  base.”  AFAM  now  requires  as 
a  minimum  an  IBM  compatible  386 
computer  with  a  4-megabyte  (MB)  RAM; 
60  MBs  of  hard  disk  space;  a  VGA  monitor 
and  a  mouse  that  is  preferred  over  the  use 
of  a  keyboard.  The  minimum  software 
requirements  are  Microsoft  Windows  3.1  or 
Microsoft  Windows  NT.  Based  on  these 
hardware  requirements,  the  increasingly 
popular  use  of  a  local  area  network  (LAN) 
provides  easy  .access  to  AFAM..  The. 
AFAM  Program  Office  distributes  the 
model  to  users  on  File  Transfer  Protocol 
(FTP),  CD-ROM,  or  3.5”  high  density 
disks.  The  availability  of  AFAM  is  limited 
only  to  the  users  hardware  capabilities. 


AFAM  Features 

The  AFAM  identifies  and  organizes  more 
than  5,000  tasks  and  sub  tasks  covering 
functional  processes  as  task  breakdown 
structures  (TBS)  within  the  six  acquisition 
phases;  pre-concept  exploration,  concept  & 
exploration;  demonstration  &  validation, 
engineering  &  manufacturing  development; 
production  &  deployment,  and  operations 
&  support.  It  describes  each  task 
associated  with  a  particular  acquisition 
process  in  common  layman  terms.  Details 
for  each  task  include  information  such  as 
required  inputs  and  outputs,  approximate 
timelines,  and  resources  required  to 
complete  the  task. 

“We  couldn’t  assume  anything,”  said  Col. 
Ferrell.  “We  took  the  approach  that  if  a 
certain  task  consistently  takes  13  steps,  our 
people  should  know  what  they  are,  how 
much  time  each  one  takes,  the  resources 
needed  to  complete  them,  and  the  pitfalls  to 
avoid.  That  doesn’t  mean  that  every 
person  will  get  from  point  A  to  point  B 
using  the  same  route,  but  at  least  they’ll  all 
have  the  same  road  map.” 

To  illustrate,  a  user  is  assigned  a  specific 
task  to  accomplish.  They  may  ask 
themselves  “Where  do  I  start?”  The 
model’s  software  displays  five  methods  to 
look  up  tasks  within  six  acquisition  phases. 
They  are:  Keyword  Search,  Document 
List,.  Graphical,.  TBS.  Hierarchy,,  and  TBS 
List. 

•  Keyword  Search  -  is  the  capability  to 
look  up  a  familiar  word  or  phrase  that 
is  common  to  a  specific  functional  area, 
for  example,  “acquisition  plan.” 

•  Document  List  -  are  documents 
identified  as  required  documents  used 
by  the  functional  areas  to  accomplish  a 
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task,  for  example,  “acquisition  decision 
memorandum.” 

•  Graphical  -  displays  the  network  as  a 
flowchart,  illustrating  the  relationships 
between  the  tasks. 

•  TBS  Hierarchy  and  TBS  List  -  displays 

tasks  down  to  six  levels  within  a 
process,  for  example,  “5.A.1  -  Manage 
Modifications”  to  “5.  A.  1.2. 3. 5 

Maintain  Funds  Data  in  Mod  Mgt  Sys 
(MMS).” 

The  model’s  software  aligns  the  tasks  in  a 
predecessor  (before)  and  successor  (after) 
network.  This  network  allows  the  user  to 
walk  through  the  acquisition  and  support 
processes.  Figure  1  displays  the  Task 
Relationships  screen  which  is  the  entrance 


window  for  accessing  task  information  or 
to  examine  predecessor  and  successor  tasks. 
This  screen  is  the  master  key  to  open  the 
door  to  the  details  of  a  task.  Simple 
clicking  the  Description,  References, 
Wisdom,  Tools,  or  Timeline  buttons 
displays  key  information  about  the  task. 
The  Local  Info  button  is  reserved  for  users 
to  store  information  related  to  their  system. 

Each  of  the  5,000  tasks  provides  key 
acquisition  and  sustainment  information  in 
the  following  areas  in  the  form  of  a  “detail 
button”  found  on  the  Task  Relationship 
screen  shown  in  Figure  1 : 

•  Task  Descriptions  --  each  task  is 
described  briefly  in  common  terms  in 
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Figure  1 .  Task  Relationship  Screen 
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one  or  two  paragraphs.  Some  are  two 
or  three  pages.  Task  inputs  and  output 
are  included  here.  Inputs  are  those 
things  (documents,  information)  from 
other  processes  that  are  required  to 
complete  the  task  at  hand.  Outputs  are 

-  — those  products  of  the  subject  task. 

Office  symbols  and  phone  numbers  of 
organizational  “experts”  are  also  listed 
in  the  description  files. 

•  References  —  brief  synopses  of  each 
directive,  regulation,  instruction,  and 
policy  letter  that  applies  to  that  task. 
When  possible,  each  referenced  item 
provides  the  user  the  appropriate 
paragraph  number.  Many  believe  this 
is  one  of  the  best  features  of  the  model. 
Also,  documents  in  the  AFAMSUP 
have  a  direct  link  in  this  area  to  the 
supplement.  This  link  will  take  you 
directly  into  the  exact  reference  in  the 
document.  It  assists  and  guides  the  user 
to  the  right  requirement. 

•  Wisdom  —  the  heart  of  AFAM  that 
captures  the  experiences  of  AFMC’s 
most  seasoned  experts.  It  consists  of 
best  practices,  lessons  learned,  and 
expert  wisdom.  Recommendations, 
practical  advice,  and  a  compilation  of 
actions  “to  take  or  not  to  take”  are  also 
available. 

•  Tools  —  composed  of 
templates/ samples,  courses,  software 
tools  and  miscellaneous  tools  that  help 
the  user  complete  the  task.  This  display 
provides  ready  access  to  electronic 
templates/ samples  to  prepare  or  update 
a  document.  It  provides  a  synopsis  of 
available  training  for  the  task  such  as 
courses  offered  at  the  Air  Force 
Institute  of  Technology,  Air  and 
Education  Training  Command,  or 
various  other  training  facilities. 
Software  tools  information  is  included 


concerning  the  purpose  and  capabilities 
of  the  tool  that  an  individual  would  use 
during  the  task.  Finally,  the 
miscellaneous  tool  data  file  contains 
information  which  does  not  readily  fall 
into  the  other  three  “tools”  categories. 

•  Timelines  —  a  “  nominal— indication  of 
the  time  and  resources  required  to 
complete  the  task.  This  information 
can  assist  the  program  manager  in 

_ establishing  a  schedule. _ 

•  Local  Information  --  as  previously 
mentioned  is  an  area  reserved  for  users 
to  store  their  own  information  related  to 
each  task. 

Finally,  packaged  with  AFAM  is 
AFAMSUP,  a  fast  text  search/retrieval 
tool,  with  key  reference  documents  such  as 
DoD  directives,  Air  Staff  and  MAJCOM 
policy  letters,  regulations,  instructions, 
military  standards,  pamphlets,  guides  and 
handbooks.  Although  AFAMSUP  may  not 
have  all  of  the  “essential”  documents 
needed  for  a  particular  task  or  process,  it 
does  provide  a  repository  of  documentation 
in  one  location.  Users  have  found  this  key 
reference  search  and  retrieval  capability  an 
extremely  popular  aspect  of  AFAM.  The 
latest  release,  AFAMSUP  Version  2.1 
contains  170  reference  documents.  It  is 
updated  semi-annually  and  with  each 
release  additional  documents  are  added. 
Like  AFAM,  it  is  distributed  in  a  Windows 
environment,  giving  the  user  more 
capabilities  and  familiarity.  AFAMSUP 
makes  research  easier  and  saves  time  —  a 
scarce  resource  these  days. 

Continuous  Improvement 

Future  enhancements  will  include  the  link 
capability  to  use  software  like 
“PerFormPro”  to  create  documents  for 
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specific  tasks.  This  will  make  the  model  a 
“doing”  tool  rather  than  a  reference  tool 
only.  Another  development  will  make 
AFAM  available  “on-line”  to  the  user. 
AFAM  as  a  multi-platform  application  for 
use  on  Macintosh  computers.  Another 
major  improvement  under  development  is 
the  full-text  search  capability  of  AFAM 
data.  The  user  will  have  the  ability  to 
search  the  entire  database  for  a  specific  task 
or  subject. 

Customer  requirements  review  boards 
contribute  significantly  to  the  continued 
enhancement  of  the  AFAM  processes  and 
data.  The  intent  is  to  continue  to  capture 
the  customer’s  or  the  user’s  desires  when  it 
comes  to  a  specific  process  and  the 
associated  tasks  within  each  process  to  get 
the  work  done.  The  database  continues  to 
expand  to  include  all  tasks  associated  with 
each  phase.  For  example,  the  increased 
emphasis  to  develop  the  operations  & 
support  phase  as  it  is  in  the  other  phases. 
Ultimately,  the  model  will  truly  reflect  the 
product  management  and  sustainment 
business. 


CONCLUSION 

Contrary  to  popular  belief  within  the 
acquisition  corps,  the  AFAM  is  not  just  a 
tool  to  teach  new  people  the  acquisition 
business,  though  it  is  invaluable  in  that 
role.  It’s  just  as  important  as  a  medium  for 
communicating  accurate  knowledge, 
wisdom,  tools,  policy,  processes,  and 
procedures  to  all  users.  As  an  AFAM  HQ 
AFMC  Financial  Management  (FM) 
representative,  it  is  my  task  to  ensure  that 
the  model’s  database  reflects  current  FM 
processes  and  policies  that  affect  the  FM 
community.  This  responsibility  holds  true 
for  all  functional  representatives. 

As  the  workforce  downsizes  and  we 
painstakingly  reengineer  our  processes,  the 
use  of  the  AFAM  will  create  a  significantly 
more  efficient  and  smarter  acquisition 
corps.  The  process  of  continual 
improvement  in  our  constantly  changing 
acquisition  environment  makes  AFAM  the 
ideal  tool.  The  AFAM  baseline  is  now  in 
review  by  the  DoD  Acquisition  Reform 
team  as  a  candidate  prototype  for  the  DoD 
acquisition  system. 


(Please  note  that  the  views,  opinions,  and/or  conclusions  in  this  paper  are  those  of  the  author 
and  should  not  be  construed  as  Department  of  Defense  of  other  government  agency  official  position .) 
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REQUIREMENTS  VERIFICATION 
IN  A  MIL-SPEC  FREE  ENVIRONMENT 


Todd  R.  Bosworth 
Sr.  Systems  Engineer 
Contraves  Inc. 


ABSTRACT 

A  systematic  approach  for  tracking  a 
Requirement/Verification  match-up  for  all 
contract  specification  requirements  on  a 
program  is  presented.  The  process  has 
successfully  been  used  on  hardware  systems 
delivered  to  the  Air  Force. 

The  process  is  described  from  the  early  stages 
of  a  program  to  completion  and  sign-off  of 
requirements,  with  its  relationship  to  all 
verification/test  documentation,  and  resulting 
test  reports. 

The  process  is  shown  to  strongly  adhere  to  the 
principles  of  design  to  test,  and  provides  the 
thread  of  consistency  throughout  the  project. 
Additionally,  by  following  the  process  through 
document  preparation  and  test  execution,  final 
performance  reports  are  easily  generated. 

INTRODUCTION 

In  a  procurement  world  with  reduced  reliance 
on  military  specifications,  it  becomes  necessary 
to  establish  an  understanding  between  the 
contracting  agency  and  the  contractor  to 
assure  that  the  procurement  successfully 
provides  the  desired  system.  This 
understanding  is  necessary,  for  both  the 
customer  and  the  vendor  to  avoid  costly 
rework,  delays,  and  overruns  that  result  from 
system  development  that  does  not  meet 
specification  intentions  beyond  the  actual 
wording  of  the  requirement. 

Contraves  has  utilized  a  system  of 
requirements  traceability  on  several  programs 
both  with  government  contracting  agencies  as 
well  as  corporate  customers  that  provides 


visibility  to  the  test  and  verification 
requirements  early  in  the  program  to  assure 
that  the  system  performs  as  expected  from  the 
beginning  of  the  design  phase.  Reflecting  the 
verification  approach  back  to  the  customer 
provides  visibility  into  our  interpretation  of 
the  intent  of  the  requirement,  as  well  as  the 
priority  of  the  requirement.  This  has  resulted 
in  the  delivery  of  hardware  systems  that  fully 
meet  customer  expectations. 

The  approach  used  has  the  positive  side  effect 
of  embodying  principles  of  “design  to  test,” 
“concurrent  engineering,”  and  incremental 
sign-off  of  contract  requirements  verification. 
This  approach  becomes  a  valuable  tool  to  be 
included  in  the  System  Engineering 
Management  Plan  (SEMP)  and  can  be  used 
with  a  large  variety  of  contract  types  including 
system  development  contracts. 

As  stated  in  IDA  (Institute  for  Defense 
Analysis)  Report  R-338,  ‘The  role  of 
Concurrent  Engineering  in  Weapons  Systems 
Acquisition:”  “The  mutually-defined,  time- 
phased  verification  actions  contained  in  the 
Systems  Engineering  Master  Schedule  provide 
the  means  for  risk  management  of  contractor 
progress.  They  are  comparable  or  superior  to 
those  currently  provided  by  the  system  of 
military  specifications  and  standards” 
(Appendix  D,  section  D.5).  This  paper 
describes  a  process  that  assures  a  suitable 
verification  program  is  followed. 

This  process  is  being  followed  by  Contraves 
with  the  delivery  of  a  3.67  meter  reflective 
telescope  to  the  Air  Force  Phillips  Laboratory 
for  installation  on  Mt.  Haleakela,  Hawaii.  It 
was  used  previously  on  a  3.5  meter  telescope 
gimbal  system  delivered  to  Phillips  Laboratory 
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which  is  now  in  operation  at  the  Starfire 
Optical  Range  at  Kirtland  Air  Force  Base  in 
New  Mexico.  The  successful  application  of 
this  process  in  these  contracts  demonstrates 
the  suitability  of  the  methodology. 

REQUIREMENT  TRACEABILITY 
OBJECTIVE 

The  verification  approach  used  by  Contraves 
depends  on  matching  each  specification 
requirement  with  a  suitable  verification 
approach.  This  match-up  is  established  early 
in  the  program,  and  is  maintained  throughout 
the  system  design,  verification  documentation, 
and  test  reports  to  provide  the  backbone  for 
system  sign-off. 

In  the  case  of  the  3.67  meter  telescope 
program  being  executed  by  Contraves,  this 
matched  pair  of  Requirements/Verification  was 
established  during  the  proposal  effort  in  the 
form  of  a  Compliance/Verification  matrix 
provided  to  the  contracting  office.  This  matrix 
provided  a  focus  for  the  proposal  team  in 
assuring  that  a  verifiable  design  was  proposed. 
It  also  furnished  insight  for  the  proposal 
evaluators  into  the  suitability  and  credibility  of 
the  proposed  system.  The  detail  used  in  this 
matrix  focused  the  verification  on  the 
specification  parameter,  rather  than  the 
specification  paragraph. 

Upon  award  of  the  contract,  the  Compliance/ 
Verification  matrix  was  expanded  into 
Verification  Information  Statements  that 
provided  the  baseline  of  the  requirement/ 
verification  match-up  for  the  rest  of  the 
program. 

THE  VERIFICATION  INFORMATION 
STATEMENT 

The  cornerstone  of  the  verification  program 
discussed  in  this  paper  is  the  set  of  Verification 
Information  Statement  (VIS)  sheets.  Each 
VIS  sheet,  in  addition  to  basic  boilerplate 
information  and  space  for  customer  and 


contractor  sign-off,  shows  the  requirement/ 
verification  approach  match-up  for  a  given 
parameter. 

Creation  of  the  Verification  Information 
Statement 

An  example  of  a  VIS  sheet  is  given  in  Figure 
1 .  There  is  one  VIS  sheet  for  each  requirement 
parameter,  for  each  method  of  verification,  and 
for  each  sequence  for  the  verification.  This 
topic  is  discussed  in  more  detail  in  the  section 
below  on  the  Integrated  Test  Plan. 

The  requirements  section  is  a  direct,  verbatim 
statement  of  the  applicable  specification 
requirement.  The  requirement  presented  on 
each  VIS  sheet  reflects  only  one  parameter. 
If  there  is  more  than  one  parameter  defined  in 
the  specification  paragraph,  additional  VIS 
sheets  should  be  prepared  for  each  parameter. 
By  breaking  down  requirements  in  this 
manner,  the  method  of  verification  (Analysis, 
Test,  Inspection)  and  sequence  of  verification 
(Critical  Design  Review,  In-Process  Inspection 
or  Test,  Factory  Acceptance  Test, 
Qualification  Test,  or  Site  Acceptance  Test) 
can  be  clearly  tracked  and  incrementally  signed 
off. 

The  Verification  Approach  section  is  a  capsule 
description  of  the  intended  verification.  The 
description  does  not  get  into  detail  about  the 
pass/fail  criteria,  tooling,  or  procedure  to  be 
used  in  verification,  but  is  presented  in  enough 
detail  that  scope  of  the  verification  is  not 
ambiguous.  Additionally,  any  test  facilities, 
subcontract  or  major  tooling  requirement  can 
be  mentioned. 

Each  Verification  Information  Statement 
should  be  covered  in  no  more  than  two  pages. 

At  Contraves,  on  smaller  programs,  the 
Verification  Information  Statements  are 
generated  by  the  Systems  Engineer  for  the 
program.  On  larger  programs  such  as  the 
telescope  programs  described  in  this  paper,  the 
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VERIFICATION  INFORMATION  STATEMENT 

AEOS  3.67  Meter  Gimbal  and  Telescope  System 
Specification : 
F29601-92-C~0006 
13  April  1993 


VIS#  4.1. 1C 


1  OF  1  REV:  A  Date:  17  May  1993 


Specification  Paragraph  Title:  Clear  Aperture 

Parameter:  Clear  Aperture/  Part  inspection 

Method:  Configuration: 

Examination  Coude 1  mirror  and  secondary  mirror  coated 

optics  pieces. 


Schedule: 

F-IP 


REQUIREMENT : 


The  clear  aperture  of  the  optical  elements  and  all  of  the  telescope  system 
structure  shall  support  the  full  primary  mirror  aperture  over  the 
unvignetted  field  of  view  as  in  section  4.1.2  at  the  bent  Cassegrain, 
Nasmyth  and  Coude .  The  clear  apertures  associated  with  the  optical 
elements  are  defined  here  as  the  portion  of  an  element's  surface  over 
which  the  specified  surface  figure  .  .  .  must  be  met.  The  elements  will 
generally  be  larger  than  their  clear  aperture  to  accommodate  edge  roll¬ 
off,  bevels,  etc.  Mirrors  which  require  an  elliptical  clear  aperture  may 
be  made  circular  or  polyhedral  in  the  interest  of  cost  effectiveness 
provided  the  excess  portions  do  not  cause  vignetting  .  .  . 


VERIFICATION  APPROACH: 


The  secondary  mirrors  and  coude'  mirrors  will  be  examined  by  use  of  their 
build  records  to  verify  that  they  meet  the  drawing  requirements,  over  the 
specified  clear  aperture,  as  derived  for  each  piece  by  analysis  on  VIS 
4.1.1A. 


RESULTS:  PASSED  _  FAILED  _  RETEST 

COMMENTS  (optional) : 


COMPLIANCE  APPROVAL: 

CONTRAVES  INC  PHILLIPS  LABORATORY 


DATE: 


DATE: 


Figure  1.  Sample  Verification  Information  Statement 
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VIS  sheets  are  generated  by  the  System  Test 
Engineer,  who  works  closely  with  the  systems 
engineer  to  assure  that  the  verification 
approaches  are  consistent  with  the  system 
design  concept  and  integration  planning.  On 
some  of  the  more  obscure  requirements,  it  may 
be  necessary  for  the  system  test  engineer  to 
poll  the  detail  design  staff  for  the  verification 
approach.  In  all  cases,  all  detail  design  lead 
engineers  will  be  called  upon  to  review  and 
pass  judgement  on,  and  provide  input  for,  the 
entire  set  of  Verification  Information 
Statements. 

Review  of  VLS  Requirements 

Upon  preparation  of  the  entire  set  of  VIS 
sheets,  it  is  necessary  to  review  the  sheets  to 
see  that  the  verification  plan  is  as  desired. 
This  review  is  done  in  two  phases.  First,  the 
internal  phase  of  review  is  performed  by  the 
contractor  team  to  assure  that  the  verification 
approach  is  consistent  with  the  intended  scope 
of  the  system  development,  and  that  the 
approach  is  consistent  with  the  organization 
capabilities  and  processes.  Secondly,  the 
program  should  be  reviewed  carefully  by  the 
customer  to  assure  that  the  system  and 
verifications  are  acceptable. 

Internal  Review  of  VLS  Requirements.  As 

stated  above,  all  design  and  production  lead 
personnel  need  to  be  involved  in  the  review  of 
the  entire  verification  package  at  the  VIS  sheet 
level.  (This  is  not  a  trivial  task  on  a  major 
program.  The  3.67  meter  telescope  program  is 
defined  by  approximately  350  VIS  sheets.) 
This  review  should  occur  at  about  the  time  of 
Preliminary  Design  Review  (PDR)  so  that  the 
Integrated  Test  Plan  can  be  completed  for 
submittal  to  the  customer  prior  to  Critical 
Design  Review. 

Review  of  this  information  is  extremely  critical 
to  the  success  of  the  process.  Review  is 
neccessary  to: 


1.  Assure  that  the  approach  on  the  VIS  is 
consistent  with  the  scope  of  the  design. 
This  provides  a  forum  for  discussion  of  the 
scope  of  the  design  across  functions  to 
establish  concurrent  engineering 
communication. 

2.  Assure  that  the  functional  disciplines  are 
aware  of  the  verification  approach  and  that 
the  completed  design  should  pass  the 
requirement.  Functional  design  lead 
personnel  are  expected  to  accept 
ownership  of  the  verification  approach  at 
this  time,  and  there  is  little  excuse  for  the 
failure  of  a  test  later  in  the  program.  This 
provides  the  groundwork  for  the  “design 
to  test”  process.  In  many  cases  on  the 
3.67  meter  telescope  program,  functional 
lead  personnel  have  modified  the  design  of 
subsystems  to  assure  compliance  as  a 
result  of  this  review. 

3.  Identify  parameters  and  specifications  that 
are  difficult  or  impossible  to  meet.  These 
areas  are  identified  at  the  early  stages  of 
the  program  for  communication  with  the 
contracting  office  for  a  request  for  a 
waiver,  the  further  clarification  of  a 
requirement,  or  the  identification  of  a  risk 
area  to  provide  extra  effort,  or  extra 
development  tests  to  verify  concepts. 

4.  Alert  the  functional  lead  engineers  to  the 
verification  process.  Their  people  will  be 
called  on  later  to  assist  in  the  development 
of  detailed  plans  and  procedures. 

5.  Assure  that  the  verification  approach  is 
tailored  as  much  as  possible  to  the  normal 
operation  of  the  organization.  Keep 
verifications  consistent  with  normal 
practice  where  practical.  This  review 
should  identify  any  verification  that  is 
outside  normal  practice  for  additional 
attention  to  assure  compliance. 

Customer  Review  of  VIS  Requirements. 

Following  the  complete  review  by  the  internal 
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project  team,  the  set  of  VIS  sheets  is 
submitted  to  the  customer  for  review  as  part  of 
the  integrated  test  plan.  The  integrated  test 
plan  should  be  submitted  to  the  customer  prior 
to  the  PDR,  the  Test  Plan  should  be  presented 
in  concept  at  the  PDR,  and  the  customer 
reviews  should  be  completed  following  the 
PDR. 

Customer  review  at  this  phase  should  be 
accomplished  to: 

1 .  Establish  an  understanding  of  the 
completed  system  operation  as  shown  by 
the  verification  approach.  This  assures 
that  the  contractor  interprets  the 
specification  as  intended. 

2.  Assure  that  the  scope  of  the  verification  is 
appropriate  for  the  importance  of  the 
requirement.  Many  requirements  in  the 
specification  are  primarily  background 
requirements  of  the  system,  but  are  not  of 
primary  operational  importance.  The 
contractor  is  going  to  verify  these  types  of 
requirements  with  simplified  verifications 
to  avoid  extra  costs.  The  customer  needs 
to  concur  with  the  contractor  assessment 
of  the  requirements. 

3.  Assure  that  the  verification  sequence  is 
consistent  with  expected  performance. 
Customer  personnel  should  be  willing  to 
sign-off  requirements  at  the  time  (in- 
process,  Factory  Acceptance,  or  Site 
Acceptance)  of  the  verification. 

On  both  the  3.5  meter  and  3.67  meter 
telescope  programs,  review  of  the  VIS  sheets 
has  provided  a  forum  for  considerable  lively 
technical  discussion  on  many  aspects  of  the 
implemented  design.  It  has  been  necessary  to 
adjust  the  VIS  sheets  to  account  for  the 
mutual  agreements  reached  during  these 
review  meetings. 

The  result  of  the  review  by  all  interested 
parties  is  that  all  parties  concur  with  the  scope 


of  the  test  program,  that  it  is  within  scope  of 
the  contract,  and  that  it  will  focus  on  delivery 
of  the  expected  system. 

VERIFICATION  DOCUMENTATION 

The  set  of  Verification  Information 
Statements,  as  the  thread  of  consistency 
throughout  the  project,  becomes  the  backbone 
of  the  test  documentation.  Figure  2  shows  the 
flow  of  documentation  through  the  entire 
verification  process. 

Integrated  Test  Plan 

The  Integrated  Test  Plan  is  the  top  level 
controlling  document  for  the  verification 
program  This  document  should  be  completed 
at  about  the  time  of  the  PDR.  The  integrated 
test  plan  includes: 

1.  Verification  rules  of  engagement, 
guidelines,  definitions,  and  test  overview. 
This  information  includes  the  following: 

a.  Test  failure  anomaly  resolution  policy. 

b.  Test  procedure  real  time  redline  policy 
(if  any). 

c.  Definition  of  verification  method. 

d.  Description  of  test  plans  and 
procedures.  This  includes  guidelines 
for  procedure  sign-off,  VIS  sheet  sign- 
off,  and  data  retention. 

2.  Verification  flowdown  matrix.  This  matrix 
is  expanded  from  the  one  often  provided 
with  the  specification  to  break  down  by 
each  parameter,  each  method  of 
verification,  and  each  verification 
sequence.  There  is  in  essence,  one  matrix 
entry  for  each  VIS  sheet.  Figure  3  shows 
one  sheet  of  the  Verification  Flow  down 
Matrix  for  the  3.67  meter  telescope  plan. 
Additional  information  on  the  matrix 
shows  the  document  numbers  for  the  test 
plan  and  procedure  for  each  entry. 
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Figure  2.  Document  Flow 


3.  Requirements  document  breakdown 
diagrams.  These  diagrams  show  which 
matrix  entries  are  verified  during  specific 
tests  and  covered  by  specific  procedure 
documents.  Figure  4  shows  the  document 
breakdown  for  the  entire  verification 
program  for  the  3.67  meter  telescope 
verifications,  and  Figure  5  shows  the  VIS 
breakdown  within  the  specific  documents 
for  In-Process  Testing  only. 

Breakdown  of  the  VIS  requirements  is 
done  to  maintain  manageable  size  of  the 
individual  documents,  and  to  allow  for 
incremental  submittal  and  customer  review 
of  the  documents  as  required  by  the 
schedule. 

4.  The  full  collection  of  Verification 
Information  Statements.  This  is  usually 
attached  as  Appendix  A  to  the  test  plan.  It 
is  preceded  by  a  Revision  Status  List  to 
identify  the  current  revision  level  of  each 
VIS. 


Test  Plan/Specsfication 

The  test  plan/specification  provides  exact, 
plain  English  description  of  the  specific  tests  to 
be  performed.  These  plans  include  the  pass/ 
fail  criteria,  any  derived  requirements,  and  the 
technical  rationale  for  the  validation  of  the  test 
for  verification  of  the  requirements. 

The  test  plan/specification  is  submitted  for 
review  as  required  by  the  customer  at  the  same 
time  as  the  procedure.  The  test  plan/ 
specification  should  describe  the  test 
completely  such  that  review  of  this  document 
should  be  the  only  review  required  by  the 
customer  to  assure  that  the  test  program  is 
complete  and  suitable. 

Each  section  of  the  plan  should  reference  the 
specific  VIS  sheet  and  the  specification 
requirement.  The  discussion  in  the  Test  Plan 
describes  a  test  that  is  consistent  with  the 
scope  of  the  verification  approach  as  shown  on 
the  VIS  and  that  verifies  the  requirement 
completely.  All  pass/fail  criteria,  tooling,  and 
environmental  considerations  are  defined. 
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Figure  3.  Verification  Flowdown  Matrix  (Partial) 
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Figure  4.  Document  Breakdown 


0) 
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Figure  5.  3.67  Meter  Telescope  In-Process  Verification  Breakdown 


Customer  review  of  the  test  plan  documents 
should  be  complete  to  assure  the  actual  design 
of  the  specific  tests  is  as  expected. 

Test  Procedures 

The  test  procedures  are  step  by  step 
descriptions  of  the  necessary  operations  for 
performing  the  tests  as  defined  in  the  test  plan. 
The  detail  of  the  procedures  is  such  that  the 
test  is  repeatable,  but  excess  detail  above  that 
necessary  to  be  performed  by  a  qualified 
operator  is  not  necessary.  Copies  of  the 
applicable  VIS  sheets  are  included  in  an 
appendix  to  the  procedure,  and  are  available 
for  sign-off  at  the  completion  of  each  section. 

Test  procedures  provide  locations  for 
recording  data  and  check-off  (initial  and  date) 
of  steps,  and  sign-off  of  the  critical  parameters 
by  a  qualified  operator  or  cognizant  engineer. 
At  the  end  of  each  verification,  the  procedure 
is  signed  off  by  a  cognizant  engineer  and  the 
System  Test  Engineer  as  being  complete  and 
acceptable  as  final  verification.  Pass/fail 
criteria  are  identified  beside  each  critical  data 
entry. 

Customer  review  of  the  procedures  prior  to 
testing  is  performed  to  assure  that  the  final 
procedure  properly  executes  the  test  designed 
in  the  test  plan/specification,  and  again  that  the 
verification  is  in  accordance  with  the  approach 
as  defined  by  the  VIS  sheet. 

Customer  witness  of  the  testing  is  sometimes 
required  by  the  contract.  In  this  case,  there 
should  also  be  a  location  at  the  end  of  each 
verification  procedure  for  the  sign-off  by 
customer  witness.  In  other  cases,  customer 
witness  of  the  actual  test  may  be  optional.  In 
this  case,  there  is  no  sign-off  of  the  procedure 
by  the  customer,  but  sign-off  of  the  VIS  sheet 
is  completed  following  customer  data  review 
of  the  completed  procedure.  The  last  step  of 
each  section  of  the  procedure  calls  for  the 
sign-off  of  the  respective  Verification 
Information  Statement. 


The  data  copy  of  the  procedure  is  maintained 
in  a  loose  leaf  binder  with  space  to  add 
additional  data,  and  a  section  for  added  test 
notes  at  the  beginning. 

Test  Reports 

With  the  proper  preparation  of  procedures  and 
careful  execution  of  tests,  test  reports  are  easy 
to  prepare.  The  test  report  for  each  individual 
test  procedure  is  completed  by  providing  a 
brief  summary  of  significant  events  during 
testing  gleaned  from  the  notes  at  the  front  of 
the  procedure  and  a  summary  table  that 
compares  the  specified  requirements  for  each 
relevant  VIS  sheet  covered  with  the  actual 
results  of  the  test.  Attached  to  this  brief 
summary  is  a  copy  of  the  “as  tested” 
procedure,  notes,  and  signed  VIS  sheets. 

Summary  Test  Report 

The  Summary  Test  Report  is  a  document  that 
contains  a  checklist  of  all  VIS  sheets,  defined 
as  to  the  sequence  of  performance,  and  space 
for  recording  the  revision  level  of  each  VIS 
sheet  signed  off.  Figure  6  shows  one  page  of 
the  checklist  used  on  the  3.67  meter  telescope 
showing  the  partially  approved  level  of  the 
verification  program.  This  document  is 
prepared,  published,  and  submitted  to  the 
customer  following  the  general  concurrance  of 
all  parties  with  the  Integrated  Test  Plan. 

As  the  procedures  are  completed  and  the  VIS 
sheets  are  signed  off,  copies  of  the  completed 
VIS  sheets  are  attached  to  the  summary  test 
report  document  and  the  respective  line  item  is 
completed  and  initialed. 

Following  all  testing,  the  final  test  report 
summary  is  completed  by  preparing  a  brief 
summary  overview  of  all  testing,  and 
combining  all  the  individual  report  compliance 
tables  into  a  complete  table  of  all  VIS 
specification  values  and  actual  results.  This, 
with  the  VIS  checklist  and  signed  VIS  sheets 
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Figure  6.  Verification  Summary  Checklist 


attached,  completes  the  test  report  for 
submittal  with  the  hardware. 

PITFALLS  AND  PROBLEM  AREAS 

As  with  any  process,  care  must  be  excercised 
to  assure  proper  operation  of  the  system. 
Some  of  the  areas  of  concern  are  discussed 
below. 

Inattention 

Probably  the  worst  thing  that  can  happen  to 
this  process  is  to  not  pay  attention  to  detail. 

If  the  customer  does  not  pay  attention  to  the 
Integrated  Test  Plan  at  the  time  of  submittal, 
there  is  a  high  probability  that  the  final  product 
will  not  perform  as  expected  due  to  a 
misinterpretation  of  the  specification. 

If  the  design  team  does  not  adaquately  review 
the  VIS  sheets  at  the  first  development,  it  is 
likely  that  the  verification  may  require  more 
resouces  than  necessary,  and  may  not  fit  the 
normal  operation  practice  of  the  organization. 

Care  in  Preparation  of  the  Specification 

It  has  been  our  experience  on  several 
programs  that  the  customer’s  first  reaction  to 
the  Integrated  Test  Plan  is  that  the 
specification  does  not  really  ask  for  what  is 
desired.  This  results  in  the  revision  of  the 
specification  by  the  contracting  office  early  in 
the  program.  (It  is  better  that  this  be 
discovered  early  than  late,  so  the  process  is 
doing  its  job.)  To  avoid  this  pitfall,  it  is 
necessary  for  the  contracting  office  to 
excercise  care  in  the  development  of  the 
specification. 

Care  in  Execution  of  the  Design 

It  is  extremely  important  that  the  finished 
product  perform  as  required  by  the  verification 
approach.  Many  times,  during  internal  review 
of  the  VIS  sheets,  we  find  the  engineering  staff 


taking  notes  in  certain  areas  of  the  VIS  sheets. 
These  notes  result  from  the  initial  concept  not 
being  easy  to  verify,  or  not  being  fully 
compliant  with  the  requirements.  These  notes 
are  then  translated  into  design  modifications 
that  bring  the  system  into  compliance.  (Again, 
this  is  an  example  of  the  system  performing 
properly.)  If,  on  the  other  hand,  the  internal 
review  of  the  VIS  sheets  is  not  adaquate,  the 
design  may  end  up  being  inadaquate,  and  the 
resulting  failed  condition  will  be  extremely 
obvious.  (Again,  the  process  is  working,  now 
for  the  benefit  of  the  customer.) 

Product  Development  in  Terms  of  Mission 
Objectives 

Our  experience  to  date  with  this  process  is 
applicable  to  systems  that  are  contracted  with 
a  clearly  defined  operational  specification.  In 
these  areas,  this  process  works  extremely  well. 

Some  contracts  are  issued  with  a  “mission 
objective”  requirement,  rather  than  a 
performance  specification.  This  process  is  as 
yet  untried  on  this  type  of  contract.  This  type 
of  contract  requires  that  the  contractor  provide 
the  additional  step  of  validating  the  derived 
performance  specification  against  the  mission 
objective.  It  seems  that  the  VIS  type  process 
could  be  useful,  but  the  quantum  leap  from 
verification  of  exact  specification  requiremnts 
to  the  validation  of  mission  requirements  might 
be  difficult  for  both  customer  and  contractor 
to  make  in  one  step. 

CONCLUSION/SUMMARY 

By  using  this  process  on  the  3.5  meter 
telescope  project,  delivered  in  1992,  Contraves 
was  able  to  focus  the  verification  process  on 
detailed  specification  issues.  This  gave  the 
customer  confidence  that  upon  completion  of 
the  sign-off  of  all  the  VIS  sheets,  the  program 
was  successfully  completed.  In  this  case,  by 
the  time  of  final  system  integra  ion  on  site,  and 
completion  of  training  for  cus  amer  personnel 
in  the  operation  of  the  system,  the  actual 
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performance  of  the  Site  Acceptance  Test  to 
verify  system  performance  was  completed  with 
customer  witness  in  approximately  4  hours. 
At  this  time,  without  additional  technical 
issues  being  discussed,  the  DD-250  was 
signed,  transferring  ownership  of  the  system  to 
the  government. 

Logicon  RDA  has  been  involved  in  the 
verification  process  of  both  the  3.5  meter 
telescope  and  the  current  3.67  meter  telescope 
project  as  technical  consultant  to  the 
Government.  Engineers  involved  in  this 
process  from  Logicon  RDA  have  indicated 
that  the  VIS  system  of  requirements 
verification  provides  an  exemplary  method  of 
assuring  that  all  contract  specification 
requirements  are  compliant  and  that  nothing  is 
being  omitted  or  neglected  by  either  customer 
or  contractor  effort. 

In  our  experience  with  this  process  on  several 
additional  programs,  it  has  been  observed  that 
the  VIS  system  provides  careful  focus  of  the 
verification  on  specific  issues  to  be  delt  with 
individually,  reducing  confusion  in  the 
verification  process.  Many  first  reactions  from 
customer  personnel  to  this  process  is  that  it  is 
excessively  cumbersome  and  tedious  to 

complete  in  detail. - However,  by — the 

completion  of  the  process,  all  customers  have 
aggreed  that  it  provides  the  confidence  that  the 
delivered  system  is  as  ordered,  and  that  when 


all  the  VIS  sheets  are  signed,  the  requirements 
are  completed. 

In  the  case  of  the  3.67  meter  telescope  now 
being  completed,  the  project  management  and 
technical  people  for  the  customer  contracting 
office  have  changed  twice  since  the  inception 
of  the  program.  The  mutual  acceptance  of  the 
VIS  process  has  provided  a  common 
denominator  for  the  program  to  prevent 
excessive  cost  growth  and  maintain  the 
program  with  minimal  scope  expansion. 

Contraves  has  found  this  process  to  be 
extremely  effective  in  delivering  hardware  to 
the  customer  that  meets  their  expectations. 
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ABSTRACT 


Professors  in  the  Test  and  Evaluation 
Department  at  the  Defense  Systems 
Management  College  have  the  opportunity 
to  work  with  test  and  evaluation 
professionals  from  all  the  services.  One 
common  area  of  contusion  is  how  to 
establish  required  linkages  between 

Department  of  Defense  (DoD)  mandated 
acquisition  documents: 


-  Test  and  Evaluation  Master  Plan 
(TEMP). 

-  Acquisition  Program  Baseline 

(APB)  (Includes  Concept, _ 

Development  and  Production 
Baselines). 

In  addition  to  preparing  the  mission  need 
statement  (MNS),  the  user  community 
prepares  the  ORD.  Each  service  designates 


-  Cost  and  Operational 
Effectiveness  Analysis  (COEA). 

-  Operational  Requirements 
Document  (ORD). 

which  agency  or  staff  prepares  the  COE  A. 
Acquisition  professionals  prepare  the  APB 
and  enabling  contractual  instruments  such  as 
system  performance  specifications  to  procure 
a  new  system. 


BY: 

LTC  Ed  Jones,  USA 
L  TC Larry  Dantnan,  USAF 
Paid  Alfieri,  USN,  CDR(ret) 
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INTRODUCTION 


Figure  1  (Documentation  Linkage) 

Although  acquisition 
professionals  have  the  lead  in 
preparing  the  TEMP,  user 
representatives  also  play  a  major 
role  in  its  preparation  by 
participation  in  the  test  integration 
and  planning  working  groups. 

(TIWGs  &  TPWGs)  The 
Operational  Test  Agencies  (OTA) 
and  service  designated  combat 
developers  often  write  the  portions 
of  the  TEMP  that  pertain  to 
operational  testing.  During  the 
acquisition  of  a  new  system,  the 
TEMP  provides  a  linkage  between 
those  documents  that  the  user 
prepares  and  those  that  the 
acquisition  community  prepares. 

Figure  1  depicts  the  key  acquisition 
documents,  other  key  supporting 
documents  and  their  linkages. 


Figure  2  depicts  the  acquisition  cycle  and  the 
decision  points  where  the  key  acquisition 
documents  are  reviewed.  The  MNS  is 
prepared  before  milestone  0.  The  APB, 


ORD,  TEMP  and  COEA  are  prepared  during 
phase  0.  These  key  acquisition  documents 
are  updated  and  reviewed  by  decision  makers 
at  each  subsequent  milestone. 
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Figure  2  (Acquisition  Process) 


Acquisition  Milestones  &  Phases 


PHASE  0  _ 
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In  a  March  1992  memorandum,  the  Under 
Secretary  of  Defense  for  Acquisition 
mandated  the  following: 

1.  Linkage  should  exist  between 
the  COEAs  and  test  and 
evaluation.  Particular  emphasis 
must  be  placed  on  linkage 
between  measures  of 
eflectiveness/performance  and 
performance  parameters  that 
define  military  utility  of  a  system. 
In  particular,  the  MOEs,  MOPs, 
and  criteria  in  key  acquisition 
documents  are  to  be  consistent. 

2.  The  TEMP  should  document 
how  the  COEA  MOEs  and 
related  MOPs  will  be  addressed  in 
test  and  evaluation. 

Performance  measurements  (or 
parameters)  used  in  the  Cost  and  Operational 
Effectiveness  Analysis  (COEA)  should  have 


corresponding  system  specific  performance 
parameters  that  are  tested  and  evaluated 
during  developmental  and  operational  tests. 
The  milestone  decision  authority  (MDA) 
must  ensure  that  the  proposed  system 
remains  the  most  cost  effective  alternative. 

If  a  proposed  system  fails  to  meet  any  of  the 
specified  performance  thresholds  in  a  COEA, 
it  is  possible  that  another  alternative  may  be 
more  cost  effective.  Changes  in 
requirements  and  associated  parameters  in 
the  Operational  Requirements  Document 
(ORD)  also  have  the  potential  to  invalidate 
the  results  of  the  COEA.  When  an  updated 
COEA  indicates  that  a  system  under 
development  is  no  longer  the  most  cost 
effective  alternative,  the  milestone  decision 
authority  (MDA)  must  determine  whether  to 
terminate,  to  modify  or  to  continue  the 
program  in  development.  The  performance 
parameters  and  issues  that  are  tested  should 
be  directly  derived  from  the  required 


639 


capabilities  in  paragraph  four  of  the  ORD. 
The  key  parameters  tracked  in  the 
acquisition  program  baseline  (APB)  should 
be  selected  from  the  same  key  performance 
parameters.  If  the  parameters  in  the  APB  are 
not  in  the  other  key  acquisition  documents, 
the  milestone  decision  authority  may  have 
little  or  no  supporting  test  data  to  evaluate 
the  APB  key  parameters. 

This  article  clarifies  guidance  from 
the  DoD  5000  series  publications  and 
provides  hints  (hopefully  helpful)  on 
implementation.  Present  performance 
parameter  terminology  used  throughout  this 
article  is  defined  in  an  endnote.1  We  will 
recommend  a  simplified  performance 
parameter  terminology  that  has  parameters 
that  are  inherently  linked  to  each  by 
definition.  Today,  acquisition  professionals 
must  infer  linkages  among  performance 
parameters.  We  will  propose  a  process  to 
establish  and  maintain  performance 
parameter  linkages  among  the  key 
acquisition  documents.  Some  of  the 
recommended  changes  may  be  appropriate 
for  formal  inclusion  in  the  next  revision  of 
the  DoD  5000  series  publications.  The  goal 
of  these  proposed  changes  and 
simplifications  is  to  ensure  that  "truly 
critical"  'system  performance  parameters  are 
correctly  specified  and  consistently  applied 
throughout  the  key  acquisition  documents. 
"Truly  critical"  parameters  (as  used  in  this 
article)  are  defined  to  be  those  parameters 
with  thresholds  that  must  be  met  for  a 
system  to  successfully  perform  a  mission 
essential  task. 

To  accomplish  the  preceding  stated 
goals,  this  article  will: 

(1)  Clearly  state  the  performance 
parameter  specification  and  linkage  problem 
and  its  impact  on  the  acquisition  system. 


(2)  Review  current  performance 
parameter  terminology  and  implementing 
policies  in  the  DoD  5000  series  publications. 

(3)  Simplify  the  DoD  5000  series 
performance  parameter  terminology  and 
establish  inherent  performance  parameter 
linkages  among  the  key  acquisition 
documents. 

(4)  Demonstrate  how  to  specify 
performance  parameters  and  to  ensure  their 
linkage  among  key  acquisition  documents. 

STEP  1 

PROBLEM  STATEMENT 

What  are  the  "truly  critical" 
performance  parameters  that  are  to  be 
specified  in  the  key  acquisition 
documents?  What  linkages  exist  between 
these  performance  parameters  among  the 
key  acquisition  documents?  Performance 
parameters  that  lack  precision  and  are  not 
consistently  specified  among  the  key 
acquisition  documents  are  a  "formula  for 
disaster".  This  "formula  for  disaster" 
consists  of  the  following  three  key 
ingredients: 

(1)  UNCERTAINTY  AS  TO 
MIA  T  PERFORMANCE  IS 
"TRULY  CRITICAL”  IN 
SYSTEMDESIGN:  When 
performance  parameters  lack 
precision  and  are  inconsistently 
specified,  the  system  engineer  must 
decide  what  is  “truly  critical”  and 
what  is  “nice  to  have”  when  writing 
the  contractual  system  performance 
specifications.  This  practice  may 
result  in  an  unstable  design  process. 
System  engineers  specify  “truly 
critical’1  performance  parameters  to 
be  key  design  objectives  to  which 
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they  optimize  the  overall  system 
design.  Inconsistently  specified 
performance  parameters  among  the 
key  acquisition  documents  promote 
confusion  as  to  which  performance 
objectives  are  central  to  the  system 
design.  As  the  “truly  critical”  design 
parameters  are  discovered,  the 
system  engineer  must  enact 
engineering  change  proposals  to 
ensure  required  performance  or 
characteristics  are  incorporated  into 
the  system  design. 

(2)  UNCERTAINTY  AS  TO  WHAT 
PERFORMANCE  IS  "TRULY 
CRITICAL  "  FOR  TESTAND 
EVAL  UA  TION  PLANNING 
Consideration  should  be  given  to  the 
decision  of  what  performance 
parameters  to  test  and  how  much 
testing  is  necessary.  Inconsistent  or 
imprecise  performance  parameters 
may  result  in  testing  the  wrong 
parameters.  Tests  that  produce  data 
to  evaluate  less  than  “truly  critical” 
performance  parameters  not  only 
waste  resources,  but  also  diverts  the 
decision  maker’s  attention  from 
evaluating  the  more  important  “truly 
critical”  performance  parameters. 
Imprecision  and  inconsistency 
among  performance  parameters 
make  it  difficult  to  determine  which 
tasks  and  performance  parameters 
are  “truly  critical”  and  for  which 
decisions? 

(3)  UNCERTAINTY  AS  TO 
mi  A  T  PERFORMANCE  is 

[ 'TRUE  Y  CRITICAL  ”  DURING 
MILESTONE  DECISIONS:  The 
key  parameters  that  are  listed  in  the 
acquisition  program  baseline  (APB) 
have  thresholds  that  if  not  met 
require  a  reevaluation  of  alternative 
concepts  and/or  design  approaches. 


If  the  key  parameters  are  different 
from  the  parameters  listed  in  the 
TEMP,  the  milestone  decision 
authority  (MDA)  will  probably  have 
insufficient  data  to  make  sound 
decisions.  It  is  also  possible  that  the 
MDA  will  approve  a  system  to 
proceed  into  the  next  acquisition 
phase  when  that  system  fails  to 
achieve  thresholds  for  “truly  critical” 
performance  parameters. 


For  example,  the  ORD  may  list  a 
critical  system  characteristic  (CSC)  to  be 
that  a  communications  system  must  provide 
voice  communications  for  ranges  up  to  35 
kilometers  under  severe  jamming  conditions 
as  defined  in  the  system  threat  assessment 
report  (STAR).  Present  DoD  guidance 
does  not  mandate  that  each  critical  system 
characteristic  (CSC)  be  listed  in  the  TEMP 
as  critical  operational  issues  (COIs), 
minimum  acceptable  operational 
performance  requirements  (MAOPRs), 
critical  technical  parameters  (CTPs)  or  in  the 
acquisition  program  baseline  (APB)  as  key 
parameters.  If  this  CSC  is  not  specified  to 
be  a  COI,  MAOPR,  key  parameter  or  CTP, 
it  is  likely  that  test  planning  will  not  place  the 
appropriate  emphasis  and  resources  into 
generation  of  test  data  to  adquately  assess 
this  critical  system  characteristic.  The 
milestone  decision  authority  might  decide 
that  the  program  should  enter  the  next 
acquisition  phase  without  adequately 
considering  voice  communications  under 
severe  jamming.  Assuming  this  CSC  to  be 
“truly  critical”,  all  the  key  acquisition 
documents  should  include  this  CSC  as  a 
"truly  critical”  parameter.  One  central 
ingredient  is  required:  We  must  have  a 


performance  parameter  terminology  that  is 
simply  pricise,  consistent  among  key 
acquisition  documents  and  identifies  “truly 
critical ”  performance  parameters 
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STEP  2:  CURRENT  TERMINOLOGY 
&  POLICY 

The  next  step  is  to  review  the 
performance  parameter  terminology  and 
implementing  policies  and  procedures  as 

Figure  3  (Performance 


prescribed  in  the  current  DoD  5000  series 
publications.  Figure  3  lists  the  performance 
parameters  that  are  currently  used  in  the  key 
acquisition  documents. 


Parameters) 


Functional  Objective 
(FO) 

Performance  ( operational 
effectiveness  and  suitability) 
capabilities  and 
characteristics 

Critical  Operational 

Issue  (COI) 

Key 

Parameters 

Measure  of  Effectiveness 
(MOE) 

System  Performance 

Minimum  Acceptable 
Operational  Performance 
Requirement  (MAOPR) 

Measure  of  Performance 
(MOP) 

Logistics 

Critical  System 
Characteristic 

Critical  System  Characteristic 

Critical  Technical 
Performance  (CTP) 

Decision  makers  use  the  functional  objectives  (FO),  measures  of  effectiveness  (MOE),  and  the 
measures  of  performance  (MOP)  in  the  cost  and  operational  effectiveness  analysis  (COEA)  to 
predict  which  is  the  most  cost  effective  concept  to  meet  a  validated  mission  need.  On  the  other 
hand,  system  engineers,  test  managers,  decision  makers  and  others  use  the  critical  operational 
issues  (COI),  the  minimum  acceptable  operational  performance  requirements  (MAOPRs)  and 
critical  technical  parameters  (CTPs)  to  determine  whether  a  system  meets  the  user's  requirements. 

Except  for  the  critical  system  characteristic  (CSC),  each  key  acquisition  document  uses 
uniquely  defined  performance  parameter  terminology.  Although  the  DoD  5000  series  publications 
do  indicate  from  where  performance  parameters  are  to  be  derived,  they  do  not  clearly  specify  the 
inherent  linkage  relationships  between  the  various  performance  parameters.  The  list  of 
effectiveness  and  suitability  parameters  and  constraints  listed  in  the  operational  requirement 
document  (ORD)  are  the  source  for  MAOPRs.  The  critical  system  characteristics  (CSCs)  listed 
in  the  ORD  and  the  technical  performance  measures  (TPMs)  provided  in  the  systems  engineering 
management  plan  (SEMP)  are  the  source  for  critical  technical  parameters  (CTPs).  One  problem 
area  common  to  all  the  services  is  confusion  over  the  relationship  (or  linkage)  between  critical 
technical  parameters  (CTPs)  and  the  minimum  acceptable  operational  performance  requirement 
(MAOPRs).  A  wide  diversity  of  opinion  exits  as  to  when  a  critical  system  characteristic  (CSC)  is 
to  be  MAOPR  verses  a  CTP.  Those  parameters  that  can  not  be  tested  without  extensive 
instrumentation  and  under  highly  controlled  conditions  are  often  designated  to  be  CTPs.  An 
example  CTP  could  be  vehicle  mobility  under  specific  road  conditions  with  specific  grades. 

Those  parameters  that  are  highly  dependent  on  operational  conditions  are  often  designated  to  be 
MAOPRs.  An  example  could  be  the  mean  time  between  operational  mission  failure  for  a  radio. 
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Questions  to  be  answered  include:  (1)  Should  CTPs  be  defined  to  support  the  evaluation 
of  MAOPRs  and  COIs?  (2)  Does  a  CTP  best  correspond  to  the  COEA's  measure  of 
effectiveness  (MOE)  or  to  the  measure  of  performance  (MOP)?  (3)  Are  all  critical  system 
characteristics  (CSCs)  valid  candidates  to  become  critical  technical  parameters  (CTPs)?  Our 
review  of  the  DoD  5000  series  publications  produced  no  definitive  guidance  and  few  clues  as  to 
the  correct  answer  to  the  preceding  questions  and  issues.  Part  7  of  DoD  5000. 2M  (TEMP 
writing  guidance)  states:  “Discuss  the  relationship  between  the  critical  technical  parameters  and 
the  minimum  acceptable  operational  performance  requirements  in  the  Operational  Requirements 
Document."  This  relationship  is  uncertain  and  few  (we  have  not  found  any!)  TEMPs  discuss  the 
relationships  between  MAOPRs  and  CTPs.  We  will  recommend  answers  to  the  preceding 
questions,  but  first  we  should  take  a  more  detailed  look  at  the  source  for  MAOPRs,  CTPs,  and 
key  parameters.  That  source  is  the  paragraph  four  of  the  ORD. 


DoDI  5000.2  (Part  4}  cites  the  ORD  to  be  a  source  for  MAOPRs,  CTPs,  and  key 
parameters.  In  the  ORD,  the  three  categories  of  required  capabilities  are:  (1)  System 
Performance:  (2)  Logistics  and  Readiness  and  (3)  Critical  System  Characteristics.  Figure 
4  depicts  the  performance  parameter  categories  with  examples  extracted  from  DoD  5000. 2M. 

System  performance  parameters  such  Figure  4  (ORD  PARAMETERS) 

as  range  and  speed  are  not  critical  .  .  .  . 

system  characteristics,  but  are  listed  (ORD)Critical  System  Characteristics 

-in  a  separate  category4itied-"system - &  System  Performance  Parameters 

performance".  All  parameters  in 

paragraph  four  are  MAOPR  i - : - 

candidates  while  the  critical  system - »  Critical  System- Characteristics - *  SysteH1  Performance  Parami 

characteristic  (CSC)  is  the  CTP  -  ECCM  -Accuracy 

source.  Therefore,  it  is  likely  that  ~  Safety  Parameters  -  Payload 

some  MAOPRs  and  CTPs  will  be  "  Electromagnetlc  ““patiability  -Speed 

derived  from  the  same  critical  system  ~  Sumvability(nuclear,  etc..)  -  Mission  reliability 

characteristic.  Use  of  the  technical  _  .  Logistis  &  Readiness 

performance  measures  (TPMs)  from  ,  ....  -Mission  Capable  Rate 

the  SEMP  as  a  CTP  source  Qt  .  ..  ..  -Operational  Availability 

-  Standardization  -MTBF 

complicates  the  specification  process.  -MTTR 

As  presently  defined,  the  TPM  can  be 

any  type  of  parameter  and  has  no  1 

limitations  on  sources.  TPMs  could 

include  all  the  parameters  in  paragraph 

four  of  the  ORD  or  exclude  them  all.  It 


•  System  Performance  Parameters 

-Range 

-  Accuracy 

-  Payload 
-Speed 

-  Mission  reliability 

•  Logistis  &  Readiness 

-Mission  Capable  Rate 
-Operational  Availability 
-MTBF 
-MTTR 


is  possible  that  many  (or  all  )  of  the  MAOPRs 
and  CTPs  will  be  the  same  parameter. 
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How  do  al!  the  performance  parameters,  characteristics,  requirements  and  issues  tie 
together?  Figure  5  depicts  the  linkages  implied  and  directed  by  the  current  DoD  5000  series 
publications.  Solid  arrows  depict  the  sources  for  performance  parameters.  The  dotted  line 
indicates  that  critical  system  characteristics  are  the  same  in  the  ORD  and  TEMP. 


Figure  5  ( Performance  Linkages ) 


Using  the  current  performance  terminology,  we  will  outline  the  process  to  establish  "truly 
critical"  performance  parameters. 

1.  Establish  the  functional  objectives  (FOs),  measures  of  effectiveness 
(MOEs)  and  measures  of  performance  (MOPs).  In  theory  (in  practice  all  the 
key  acquisition  documents  are  often  prepared  simultaneously),  the  COEA  is  one  of 
the  first  documents  to  be  drafted.  It  provides  analysis  to  select  a  preferred  system 
concept.  The  phase  0  COEA  is  based  on  the  mission  need  statement  (MNS)  and 
has  the  following  goals: 

.  Identify  the  performance  and  cost  advantages  and  disadvantages  between  proposed 
ystems  over  the  existing  system  and  /or  a  modified  existing  system. 

.  Broadly  define  the  system  characteristics  needed  in  the  new  system. 

.  Select  the  preferred  alternative  to  carry  into  phase  I  of  the  acquisition  cycle. 


2.  Derive  the  required  capabilities  in  the  ORD  from  the  mission  need 
statement  (MNS)  and  the  COEA.  The  MOEs  and  MOPs  are  candidates  for 
MAOPRs  and  CTPs.  The  technical  performance  measures  (TPMs)  in  the  System 
Engineering  Management  Plan  (SEMP)  (when  it  exists}  are  CTP  candidates.  In 
paragraph  four  of  the  ORD,  list  the  required  capabilities  in  the  prescribed  format. 
Identify  which  parameters  are  to  be  MAOPRs  and  CTPs. 


Figure  6  (ORD  Characteristics) 

(ORD)Critical  System  Characteristics 
&  System  Performance  Parameters 


•  Critical  System  Characteristics 
~  ECCM(MAOPR &CTP) 

-  Safety  Parameters 

-  Electromagnetic  com patiability 

-  Survivability(nuclear,  etc..) 

-  Transportability(CTP) 

-  Energy  efficiency 

-  Interoperability(CTP) 

-  Standardization^ AOPR) 


•  System  Performance  Parameters 

-  Range(MAOPR  &  CTP*) 

-  Accuracy(MAOPR) 

-  Payload(M  A  OPR  &  CTP*) 

-  Speed(M  AOPR  &  CTP*) 
-Mission  reliability(MAOPR) 

Logistis  &  Readiness 

-Mission  Capable  Rate(MAOPR) 
-Operational  Availability 
-MTBF(MAOPR) 
~MTTR(CTP*) 


CTP*-SPECIFIED  AS  TPM  IN  SEMP. 


3.  Directly  extract  the  key  parameters  from  the  ORD  into  the  APB.  The 

milestone  decision  authority  (MDA)  approves  the  specification  of  key  parameters 
in  the  APB. 

4.  Define  the  critical  operational  issues  (COIs).  COIs  are  questions  that  when 
answered  provide  a  basis  for  the  operational  test  agency  (OTA)  and  the  Director 
of  Operational  Test  and  Evaluation  (DOTE)  to  assess  operational  effectiveness 
and  suitability.  The  COEA  functional  objectives  (FOs)  are  COI  candidates.  The 
functional  objectives  (FOs)  as  defined  in  the  COEA  are  the  mission  essential  tasks 
that  a  system  must  perform. 

The  following  recommended  hints  can  be  used  to  establish  parameter  linkages  and  to 
improve  the  overall  performance  parameter  specification  process: 
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a.  Form  an  integrated  product  team  to  coordinate  and  draft  the  key  acquisition 
documents.  During  step  4,  we  will  provide  a  detailed  team  composition  and  team  charter. 

b.  Expand  the  critical  technical  parameter  matrix  (presently  required  in  the  TEMP)  to 
include  the  MAOPRs.  In  the  past,  this  table  only  tracked  developmental  testing.  Combining  key 
operational  and  development  test  events  into  one  matrix  provides  test  planners/managers  and 
decision  makers  a  more  complete  and  integrated  status  report  on  how  the  system  is  performing. 
Figure  7  depicts  an  example  matrix. 


Figure  7  (CTP  &  MAOPR  Test  Status) 
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Availability 

TX 
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c.  Use  the  following  guidelines  to  specify  MAOPRs  and  CTPs: 

(1)  Designate  top  level  performance  and  suitability  parameters  to  be 
MAOPRs.  Top  level  means  that  the  parameter  measurement  is  not  used  to 
calculate  or  predict  another  parameter  measurement.  For  example,  probability  of 
shooting  down  an  incoming  enemy  aircraft  could  be  a  MAOPR. 

(2)  Designate  second  level  performance  parameters  that  support  the 
calculation  or  prediction  of  a  MAOPPv  to  be  a  CTP.  For  example,  probability  of 
radar  detection  and  probability  of  successful  missile  launch  could  be  critical 
technical  parameters  that  support  the  prediction  of  a  MAOPR  for  probability  of 
shooting  down  an  incoming  enemy  aircraft. 

(3)  Do  not  restrict  specification  of  MAOPRs  to  those  parameters  that  can 
be  adequately  tested  in  operational  testing  and  CTPs  to  those  parameters  that  can 
be  adequately  tested  in  developmental  testing.  Problems  in  testing  MAOPRs  and 
CTPs  can  be  adequately  addressed  in  the  test  limitations  part  of  the  test  report. 

d.  Ensure  that  each  MAOPR  supports  the  assessment  of  one  or  more  COIs.  A 
MAOPR  that  does  not  support  a  COI  indicates  a  problem.  Either  the  MAOPR  is 
not  a  "truly  critical"  parameter  or  a  COI  is  missing  and  needs  to  be  assigned. 
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e.  Although  presently  not  required  in  the  ORD,  parameter  linkages  among  the 
COIs,  MAOPRs,  and  CTPs  should  be  depicted  in  paragraph  four.  An  example 
method  to  depict  parameter  linkages  is  to  use  a  three  digit  numbering  system. 

For  example,  CTP  3-2-1  denotes  CTP  number  1  that  supports  the  calculation  of 
MAOPR  2  that  supports  the  assessment  of  COI 3.  An  expanded  example  of  the 
recommended  numbering  system  is  below: 

COI  1 :  Does  SINCGARS  provide  reliable  voice  communications  with  adequate  range  to 
support  command  and  control  for  forward  deployed  combat  forces  at  division  level  and 
below? 


MAOPR  1-1:  Aggregate  voice  message  completion  rate  (MCR)  within  a  division 
with  jamming.  {Threshold  80%;  Objective  90%.} 

{NOTE:  MAOPR  1-1  is  a  weighted  average  of  CTP  demonstrated  values.) 

CTP  1-1-1:  Voice  MCR  at  10,  20,  25,  30,  35  Kms  with  severe  jamming  levels. 
{Threshold  50%;  Objective  55%.} 

CTP  1-1-2:  Voice  MCR  at  10,  20,  25,  30,  35  Kms  with  moderate  jamming 
{Threshold  80%;  Objective  90%.} 

CTP  1-1-3:  Voice  MCR  at  10,  20,  25,  30,  35  Kms  with  low  jamming 
{Threshold  85%;  Objective  95%.} 

f.  To  assess  whether  parameter  linkages  exist  between  the  TEMP  and  the  COEA, 
the  test  manager  should  construct  a  parameter  correlation  matrix.  This  matrix 
should  be  added  as  an  appendix  to  the  TEMP. 


Figure  8  (Parameter  Correlation  Matrix) 
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STEP  3:  SIMPLIFICATION  &  LINKAGE 


Enough  discussion  of  the  complications,  unanswered  questions,  and  imprecise  definitions! 
What  should  be  done?  We  need  a  "Brave  New  World"  in  the  establishment,  tracking,  and  linking 
of  performanceparameters.  A  new  way  of_ doing,  business  is  required  The  key  is  to  consistently 
apply  a  simplified  performance  parameter  terminology  that  is 
common  (or  inherently  linked)  among  all  of  the  key  acquisition 
documents.  Whenever  possible,  the  parameters  should  be  same. 

First  we  must  simplify  the  DoD  5000  series  performance  parameter 
(and  issue/task)  terminology. 

The  critical  operational  issue  (COI)  provides  criteria  to 
assess  operational  effectiveness  and  suitability  as  demonstrated 
during  an  Initial  Operational  Test  and  Evaluation  (IOTE).  COIs  are 
normally  questions  that  have  a  yes  or  no  answer.  Unlike  the  MAOPR  and  CTP,  the  COI  has  no 
DoD  5000  series  mandated  sources.  Its  derivation  is  highly  reliant  on  the  good  judgment  of  the 
writer.  The  TEMP  is  the  only  key  acquisition  document  to  list  the  COIs.  Because  the  COI  lacks 
inherent  relationships  with  other  performance  parameters  and  its  derivation  source  is  uncertain,  it 
does  not  meet  our  standards  for  simplicity  and  inherent  linkage.  We  will  replace  the  critical 
operational  issue  (COI)  with  a  new  top  level  performance  requirement  criterion  that  is  named  the 
critical  operational  task  (COT). 


New  TOP  Level  Performance  Parameter  Definition: {ORD,  TEMPI 

CRITICAL  OPERATIONAL  TASK  (COT)-  A  COT  is  a  statement  describing, 
in  quantitative  or  qualitative  terms,  a  top  level  mission  essential  task  that  a  specific 
system  will  be  expected  to  perform  in  order  to  satisfy  a  task  implied  or  explicitly 
stated  in  a  Mission  Need  Statement  (MNS).  The  overall  system  effectiveness  and 
suitability  is  evaluated  by  assessing  the  degree  to  which  the  COTs  are  successfully 
accomplished.  Critical  operational  parameters  (COPs)  are  specified  to  assess  the 
degree  to  which  a  COT  is  accomplished.  COTs  correspond  to  the  non-system 
specific  functional  objectives  (FO)  in  the  COEA. 


The  system  specific  critical  operational  task  (COT)  corresponds  to  the  non-system  specific 
functional  objective  (FO)  in  the  COEA.  The  definition  for  a  FO  follows: 

New  TOPLeve[  Perfomiance  Parameter  Definition: fCOEA} 

FUNCTIONAL  OBJECTIVEfFOt-  A  FO  describes,  in  quantitative  or 
qualitative  terms,  a  top  level  mission  essential  task  that  the  alternative  systems  will 
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be  expected  to  perform  in  order  to  satisfy  a  task  implied  or  explicitly  stated  in  a 
Mission  Need  Statement  (MNS).  It  is  stated  to  be  applicable  to  all  alternatives 
that  are  being  considered  in  the  Cost  and  Operational  Effectiveness  Analysis 
(COEA).  Measures  of  effectiveness  (MOEs)  are  specified  to  assess  the  degree  to 
which  a  FO  is  accomplished 

We  have  already  discussed  the  problems  that  the  services  have  encountered  in  using  the 
minimum  acceptable  operational  performance  parameter  (MAOPR)  and  the  critical  technical 
parameter  (CTP).  We  will  replace  the  MAOPR  with  a  new  performance  parameter  that  is  named 
the  critical  operational  parameter  (COP).  The  critical  operational  parameter  (COP)  is  a  system 
specific  measure  of  effectiveness  or  suitability  that  is  defined  to  assess  a  COT 


New  Second.  Level  Perfomumce  Parameter  Definition: (ORD,  TEMP,  APBl 

CRITICAL  OPERATIONAL  PARAMETER  tCOPI  -  A  COP  is  a  quantitative 
or  qualitative  measure  of  physical  performance  or  suitability  that  indicates  the 
degree  to  which  a  specific  system  successfully  performs  a  critical  operational  task 
(COT)  under  specified  conditions.  Each  COP  should  normally  have  specified 
thresholds  and  objectives  and  will  normally  correspond  to  a  measure  of 
effectiveness  (MOE)  in  the  COEA. 


COPs  provide  data  and  information  to  assess  whether  a  system  can  perform  a  COT. 

Using  test  results  in  which  COPs  were  tested,  the  decision  maker  makes  an  informed  judgment  as 
to  whether  a  COT  is  successfully  performed.  The  COT  does  not  normally  have  associated 
thresholds  and  objectives.  The  COP  replaces  the  MAOPR  and  CTP  (as  previously  defined). 
Whenever  possible,  COPs  are  tested  during  both  operational  and  developmental  testing.  The 
ideal  situation  is  to  test  all  "truly  critical1*  performance  parameters  under  operational 
conditions.  The  tester,  engineer,  and  user  ultimately  want  to  know  how  the  system  will  work 
during  combat,  not  in  a  lab.  Unfortunately,  technical  limitations  (required  instrumentation  etc.) 
and  resource  constraints  often  limit  the  extent  to  which  testing  can  be  operational.  Today,  CTPs 
are  normally  defined  to  test  those  parameters  that  are  to  be  tested  in  developmental  testing.  We 
will  no  longer  make  a  distinction  in  specifying  a  parameter  based  on  which  type  of  testing  to  be 
used  to  assess  that  parameter.  We  want  to  know  how  a  system  performs  under  operational 
conditions.  Some  COPs  that  require  extensive  instrumentation  and  highly  controlled  test 
conditions  can  only  be  adequately  tested  during  developmental  testing.  In  this  case,  the  evaluator 
must  extrapolate  the  developmental  test  results  into  an  operational  scenario.  For  major  program 
milestone  decisions,  the  Director  of  Operational  Test  and  Evaluation  (DOTE)  with  input  from  the 
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Director  Test,  System  Engineering  and  Evaluation  (DTSEE)  and  the  services'  Operational  Test 
Agencies  must  approve  to  what  extent  COPs  can  be  evaluated  using  results  from  less  than  folly 
operational  testing.  These  limitations  must  be  listed  as  a  test  limitation  in  the  test  report  and  in 
the  system  evaluation.  Whenever  practical,  the  COPs  are  to  be  tested  during  the  dedicated  IOTE 
to  verify  test  results  from  earlier  developmental  and  operational  testing.  When  testing  COPs 
during  a  dedicated  IOTE,  the  title  10  (USC)  constraints  on  using  contractors  REMAINS  in 
effect.  The  "Brave  New  World"  recommends  changes  to  the  DoD  5000  series  publications. 
Recommended  changes  in  law,  although  desired,  are  not  required. 

The  measure  of  effectiveness  (MOE)  in  the  COEA  corresponds  to  the  system  specific 
COP.  It  is  defined  as  follows: 

New  Second  Level  Performance  Parameter  Definition: {COEA} 

MEASURES  OF  EFFECTIVENESS  (MOE)  -  A  MOE  is  a  quantitative  or 
qualitative  measure  of  physical  performance  or  suitability  that  indicates  the  degree 
to  which  a  system  successfully  performs  a  functional  objective  (FO)  under 
specified  conditions.  Each  MOE  should  normally  have  specified  thresholds  and 
objectives.  The  MOE  is  applicable  to  all  alternative  systems  that  are  being 
considered  during  the  COEA.  The  MOE  corresponds  to  the  critical  operational 
parameter  (COP)  which  is  system  specific. 

A  performance  parameter  that  is  defined  to  support  the  assessment  of  a  critical  operational 
parameters  (COPs)  during  test  and  evaluation  is  designated  to  be  a  critical  parameter  (CP).  The 
CP  includes  both  system  performance  and  supportability  parameters.  It  is  tested  during 
developmental  and  operational  testing  in  the  same  manner  as  previously  discussed  for  the  COP. 
When  quantitative,  the  CP  is  a  number  that  is  used  to  calculate  the  COP.  For  example,  the  mean 
time  between  failure  (a  possible  CP)  is  used  to  calculate  the  operational  availability  (a  possible 
COP).  When  qualitative,  the  CP  provides  essential  information  to  be  used  to  assess  a  COP.  For 
example,  statements  from  soldiers  on  ease  of  use  could  be  used  in  the  assessment  of  a  COP  which 
addresses  the  man  to  machine  interface.  The  CP  corresponds  to  the  measure  of  performance 
(MOP)  in  the  COEA.  Definitions  for  the  CP  and  MOP  follow: 

Third  Level  Performance  Parameter  Definition:  10 RD,  TEMP,  APBl 

CRITICAL  PARAMETER  (CP)  —  A  CP  is  a  system  specific  quantitative  or 
qualitative  measure  of  physical  performance  or  suitability.  It  is  specified  to 
support  the  assessment  of  a  critical  operational  parameter  (COP).  It  normally  has 
a  threshold  and  objective.  The  CP  corresponds  to  the  MOP  in  a  COEA. 

Third  Level  Performance  Parameter  Definition: (COEA  l 

Measure  of  performance  (MOP)  —  A  MOP  is  a  non-system  specific 
quantitative  measure  of  physical  performance  or  suitability.  It  is  specified  to 
support  the  assessment  of  a  measure  of  effectiveness  (MOE)  in  the  COEA.  It 
normally  has  a  threshold  and  objective.  The  MOP  corresponds  to  the  CP  in  the 
ORD,  TEMP  and  APB. 
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The  COTs,  COPs,  and  CPs  that  are  in  the  TEMP,  ORD,  and  APB  are  now  inherently 
linked  to  each  other  and  to  the  FOs,  MOEs  and  MOPs  that  are  in  the  COEA.  In  the  DoD  5000 
series  publications,  all  other  performance  parameter  terminology  should  be  purged.  Milestone 
decision  authorities  (MDA)  must  refrain  from  adding  performance  parameters  to  the  APB  that  are 
not  approved  COPs  or  CPs  in  the  ORD  and  TEMP.  Figure  9  depicts  the  new  performance 
parameter  terminology  with  inherent  linkages.  Solid  lines  denote  the  source  for  a  lower  tier 
parameter.  The  dotted  line  denotes  a  lateral  tier  linkage  between  a  COEA  parameter  and  a 
parameter  used  in  the  ORD  and  TEMP. 

Figure  9  (New  World) 

BRAVE  NEW  WORLD 


COEA  TEMP  &  ORD 


1.  COT,  COP  &  CP  ARE  SYSTEM  SPECIFIC. 

2.  FO,  MOE  AND  MOP  APPLY  TO  ALL  COEA  ALTERNA  TIVES 


Key  parameters  in  the  APB  are  defined  as  stated  below: 

New  Definition: 

KEY  PARAMETERS  -  Those  critical  operational  parameters  (COPs)  and 
critical  supporting  parameters  (CPs)  that  have  thresholds  that  when  not  met,  may 
require  a  reevaluation  of  alternative  concepts  and/or  design  approaches.  Key 
parameters  are  listed  in  the  acquisition  program  baseline.  On  an  exception  basis, 
the  milestone  decision  authority  may  uniquely  define  a  key  parameter.  When  this 
happens,  this  parameter  should  be  considered  for  inclusion  as  a  COP  or  CP  in  the 
TEMP. 
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To  accommodate  the  simplified  performance  terminology,  we  recommend  the  below 
revised  ORD  format  for  paragraph  four. 


Figure  10 (ORD  FORMAT) 


ORD  FORMAT 

PARAGRAPH  4  (REQUIRED  CAPABILITIES) 

COT  1 

COP  1-1 

CP  1-1-1 
CP  1-1-2 
etc.. 

COP  1-2 


CP  1-2-1 
CP  1-2-2 


etc.. 

COT  U 

COT  2 

I 

COP  2-1 

U-U-th* - CP# 

etc... 

1 

COP  u 

By  now,  some  readers  may  have  concluded  that  this  all  sounds  nice,  but  our  service  or 
organization  does  not  have  the  time,  expertise  or  established  staffing  process  to  participate  in  the 
"Brave  New  World"  of  performance  parameter  establishment  and  linkage.  Unfortunately  the 
rea[  world.  il  ^ess  than  ideal.  The  COEA  is  sometimes  performed  in  a  vacuum  with  little  or  no 
contact  with  the  staffs  and  agencies  that  prepare  the  ORD,  TEMP  and  APB.  Some  may  argue 
that  the  very  nature  of  the  acquisition  process  makes  the  maintenance  of  consistent  performance 
parameters  among  key  documents  prepared  by  various  groups  scattered  over  the  United  States- 
MISSION  IMPOSSIBLE.  The  remainder  of  the  article  proposes  a  procedure  to  establish  valid 
and  consistent  performance  parameters  among  all  the  key  acquisition  documents. 
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_ Who  .is  involved  in  the  specification  and _ Figure  11  (IADDT- Team.  Composition) 

linkage  process?  An  integrated  acquisition 
document  development  team  (IADDT-see  figure 
1 1)  should  write  and  coordinate  the  required 
capabilities  parts  of  the  key  acquisition  documents 
for  a  proposed  system.  The  team  support  costs  in 
terms  of  personnel  and  TDY  costs  will  be  much  less 
than  the  cost  of  proceeding  into  later  acquisition 
phases  with  inconsistent  and  poorly  specified  key 
performance  parameters.  An  engineering  change 
proposal  (ECP)  resulting  from  a  change  in  specified 
performance  often  results  in  significant  costs.  For 
major  programs  during  phase  three  of  the 
acquisition  process,  ECP  costs  are  usually  in  the 
million  dollars  plus  bill  category! 

When  does  the  performance  parameter 
specification  and  linkage  process  happen?  The 
“when”  is  answered  by  reviewing  the  phased 
acquisition  process,  {see  figure  one}  The  phased 
acquisition  process  coupled  with  the  evolutionary 
requirements  process  complicates  the  performance 
parameter  specification  and  linkage  process.  The 
process  often  lasts  more  than  eight  years  and  the 

evolutionary  requirement  process  involves  continual  requirement/performance  parameter 
refinement.  Other  people  in  other  articles  can  describe  how  to  shorten  and  simplify  this  process. 

The  simplified  performance  parameter  terminology  and  recommended  process  proposed  in  this 
article  will  work  today  and  in  the  future  world  of  acquisition  reform.  The  integrated  acquisition 
document  development  team  (IADDT)  should  be  formed  upon  approval  of  a  mission  need 
statement  (MNS).  The  IADDT  develops  the  functional  objectives  (FOs),  measures  of 
effectiveness  (MOEs)  and  supporting  measures  of  performance  (MOPs)  to  be  used  in  the  COEA. 

Prior  to  milestone  I,  the  IADDT  drafts  the  ORD,  the  TEMP  and  APB  for  the  most  likely 
preferred  concept  (or  concepts).  All  the  key  acquisition  documents  must  be  approved  prior  to  or 
in  conjunction  with  the  milestone  I  Defense  Acquisition  Board  (DAB).  Whenever  a  FO,  MOE, 

MOP,  COT,  COP,  or  CP  is  changed;  the  IADDT  members  must  review  all  the  key  acquisition 
documents  to  ensure  performance  parameter  linkages  are  maintained.  At  a  minimum,  the  IADDT 
will  review  all  key  acquisition  documents  prior  to  a  major  test  and  prior  to  each  milestone 
decision  review.  The  IADDT  reviews  should  be  conducted  to  permit  sufficient  time  to  enact  any 
needed  changes  prior  to  a  major  test  or  milestone  review. 


POC  from  agency  that  prepares  COEAs 

Test  project  officer  from  OTA. 

Test  Manager  from  Program  Management 
Office. 

Test  and  Readiness  POC  from  Program 
Executive  Office.. 

Integrated  Logistics  Support 
representative  from  the  Materiel  Support 
Command. 

System  Engineer  from  likely  matrix 
support  agency  or  other  appropriate 
source. 

Test  facility  POCs  from  anticipated  test 
ranges/facilities. 

POC  from  PEO  or  designated  agency 
responsible  for  drafting  the  APB. 

User  representative. 
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STEP  4:  A  “HOW  TO”  EXAMPLE 


We  have  discussed  the  "what,  who  and  when".  Now  for  the  fun  part-  the  how.  Using  the 
simplified  performance  parameter  terminology  that  is  defined  in  step  3,  we  will  outline  the  key 
steps  in  the  specification  and  linkage  process  with  a  specific  example.  We  will  also  suggest  some 
tools  to  assist  the  integrated  acquisition  document  development  team  (IADDT)  in  the 
accomplishment  of  their  mission  to  consistently  specify  "truly  critical"  performance  parameters 
among  the  key  acquisition  documents. 

An  approved  mission  need  statement  (MNS)  normally  starts  the  acquisition  cycle.  During 
phase  0,  the  four  key  acquisition  documents  are  drafted  in  preparation  for  the  milestone  decision 
authority  (MDA)  to  review  for  a  milestone  I  decision.  This  is  a  critical  period  of  time  in  the 
specification  of  performance  parameters  among  the  key  documents.  Decisions  on  required  system 
performance  that  are  made  during  phase  0  significantly  impact  the  system  design  and  life  cycle 
costs.  It  is  also  the  time  of  greatest  uncertainty  as  to  what  the  "truly  critical"  critical  operational 
parameters  (COPs)  and  critical  parameters  (CPs)  may  eventually  be.  In  theory,  the  COEA  is 
prepared  for  each  approved  MNS.  The  COEA  provides  the  analysis  to  support  the  recommended 
system  alternate.  In  practice,  all  the  key  acquisition  documents  are  often  prepared  simultaneously. 
Considering  the  preliminary  results  from  the  COEA,  the  IADDT  drafts  the  TEMP,  ORD  and  APB 
for  the  likely  most  cost  effective  alternative.  If  the  analysis  does  not  clearly  support  a  best 
alternative,  it  may  be  necessary  to  draft  the  key  acquisition  documents  for  several  of  the 
alternatives  or  even  all  of  the  alternatives. 

We  will  use  the  combat  net  radio  as  an  example  system  to  demonstrate  the 
performance  specification  and  linkage  process.  For  brevity  sake,  only  one  functional  objective 
and  its  corresponding  critical  operational  task  will  be  fully  developed  in  this  example.  Three 
system  alternatives  are  proposed. 

(1)  Frequency  hopping  radio. 

(2)  Product  improved  AN/VRC-12  series. 

(3)  Improved  high  frequency  radios. 

1.  Considering  a  mission  area  analysis,  an  approved  mission  need  statement  (MNS)  starts 
the  process: 

MNS:  Replace  the  aging  AN/VRC-12  series  combat  net  radios  that  are  easily  jammed, 
have  poor  supportability/maintainability,  and  low  operational  availability.  When 
operating  against  the  current  and  projected  threat,  the  replacement  system  must 
provide  reliable  voice  communications  with  adequate  range  to  support  command  and 
control  for  forward  deployed  combat  forces  at  division  level  and  below.  The 
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replacement  system  must  also  provide  reliable  data  communications  with  adequate 
throughput  and  range  to  support  battalion  level  and  below  digital  targeting  systems 
and  command  and  control  systems  currently  in  a  division.  The  replacement  system 
must  reduce  operational  and  support  costs  and  improve  operational  availability.  The 
replacement  system  must  be  interoperable  with  other  the  services  and  equivalent  level 
NATO  ground  forces. 


2.  Use  the  following  numbering  system  for  all  FOs,  MOEs,  MOPs,  COTs,  COPs,  and  CPs. 


Parameter  Numbering  in  ORD,  TEMP,  and  APB 


Parameter  Numbering  in  COEA 


MOE  U  REFERENCED  MOE  # 


Figure  12  (Parameter  Numbering) 


3.  Derive  the  COEA  functional  objectives  (FOs)  from  the  mission  need  statement. 

MNS:  Replace  the  aging  AN/VRC-12  series  combat  net  radios  that  are  easily 
jammed,  have  poor  supportability/maintainability,  and  low  operational  availability. 
When  operating  against  the  current  and  projected  threat,  the  replacement  system  must 
{FO  1 1  provide  reliable  voice  communications  with  adequate  range  to  support 
command  and  control  for  forward  deployed  combat  forces  at  division  level  and  below. 
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The  replacement  system  must  also  {FO  2}provide  reliable  data  communications  with 
adequate  throughput  and  range  to  support  battalion  level  and  below  digital  targeting 
systems  and  command  and  control  systems  currently  in  a  division.  The  replacement 
system  must  {FO  3}reduce  operational  and  support  costs  and  improve  operational 
availability.  The  replacement  system  fFQ4)must  be  interoperable  with  the  other 
services  and  equivalent  level  NATO  ground  forces. 

4.  Specify  the  measures  of  effectiveness  (MOEs)  and  supporting  measures  of 
performance  (MOPs)  for  functional  objective  one. 

{FO  1}  provide  reliable  voice  communications  with  adequate  range  to  support  command 
and  control  for  forward  deployed  combat  forces  at  division  level  and  below. 


MOE  1-1-  Aggregate  voice  message  completion  rate  (MCR)  within  a  division  with 
projected  threat  jamming. 


MOP  1-1-1 
MOP  1-1-2 
MOP  1-1-3 
MOP  1-1-4 


Voice  MCR  at  10,20,25,30,35  Kms  with  severe  jamming. 

Voice  MCR  at  10,20,25,30,35  Kms  with  moderate  jamming 

Voice  MCR  at  10,20,25,30,35  Kms  with  low  jamming 

Voice  MCR  at  10,20,25,30,35  Kms  with  severe  jamming  and  retransmission 


sites. 


MOE  2-  Voice  message  completion  rate  (MCR)  within  a  division  with  jamming. 

MOP  2-1  :  Voice  MCR  at  10,20,25,30,35  Kms  with  no  jamming. 

MOP  2-2  :  Voice  MCR  at  10,20,25,30,35  Kms  with  retransmission  sites. 

5.  Assuming  that  preliminary  analysis  from  the  COEA  indicates  that  the  frequency 
hopping  radio  is  the  most  likely  cost  effective  system  to  meet  the  requirements  specified 
in  the  mission  need  statement,  specify  the  "truly  critical"  parameters  to  be  used  in  the 
ORD,  TEMP,  and  possibly  the  APB.  {Only  one  COP  with  supporting  COTs  will  be 
developed}.  For  this  example,  the  specification  of  the  COTs,  COPs,  and  CPs  is  trivial. 
Functional  objectives  with  supporting  MOEs  and  MOPs  translate  directly  into  COTs  with 
supporting  COPs  and  CPs. 

{COT  1}  Provide  reliable  voice  communications  with  adequate  range  to  support 
command  and  control  for  forward  deployed  combat  forces  at  division  level  and  below. 
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COP  1-  Aggregate  voice  message  completion  rate  (MCR)  within  a  division  with 
projected  threat  jamming. 

CP  1-1  :  Voice  MCR  at  10,20,25,30,35  Kms  with  severe  jamming. 

CP  1-2  :  Voice  MCR  at  10,20,25,30,35  Kms  with  moderate  jamming 

P  1-3  :  Voice  MCR  at  10,20,25,30,35  Kms  with  low  jamming 

CP  1-4  :  Voice  MCR  at  10,20,25,30,35  Kms  with  severe  jamming  and  retransmission 

sites. 


COP  2-  Voice  message  completion  rate  (MCR)  within  a  division  with  jamming. 

CP  2-1  :  Voice  MCR  at  10,20,25,30,35  Kms  with  no  jamming. 

CP  2-2  :  Voice  MCR  at  10,20,25,30,35  Kms  with  retransmission  sites. 

NOTE:  It  is  not  always  this  easy.  When  system  alternatives  are  not  similar,  the  COEA 
parameters  must  be  more  generic  in  nature  to  apply  to  all  alternatives.  For  example,  two 
alternatives  to  kill  SCUDS  could  be  surface  to  air  missiles  and  air  to  air  launched  missiles.  An 
appropriate  measure  of  effectiveness  (MOE)  might  be  the  time  required  to  bring  the  system  into  a 
fijlly  operational  status  upon  deployment  into  the  theater  to  be  protected  from  SCUD  attack.  For 
these  two  alternatives,  an  inappropriate  MOE  would  be  the  time  required  to  move  and  erect  a 
ground  missile  launcher.  This  MOE  is  only  applicable  to  one  alternative.  For  the  surface  to  air 
missile  alternate,  the  IADDT  might  specify  the  time  required  to  move  and  erect  a  ground  missile 
launcher  to  be  a  COP.  This  COP  would  then  be  linked  in  a  parameter  correlation  matrix  to  the 
COEA  MOE  that  specifies  the  time  required  to  bring  the  system  into  a  fully  operational  status 
upon  deployment  into  the  theater  to  be  protected  from  SCUD  attack. 

6.  Specify  the  key  parameters  to  be  used  in  the  APB.  COPs  and  supporting  CPs  are 
candidates  for  the  key  parameters.  The  APB  is  a  top  level  management  document  with 
and  the  list  of  key  parameters  should  remain  small.  A  general  rule  is  to  list  all  the  COPs 
and  few,  if  any,  CPs.  All  COPs  and  CPs  are  "truly  critical"  but  they  all  do  not 
necessarily  meet  the  requirement  that  a  failed  threshold  necessitates  a  reevaluation  of 
alternative  concepts  or  design  approaches.  The  IADDT  must  evaluate  all  the  COPs  and 
CPs  to  determine  which  ones  meet  the  requirements  associated  with  a  key  parameter  in 
the  APB.  Only  list  those  CPs  that  are  known  to  have  high  technical  risk  and  clearly 
dominate  the  threshold  of  the  supported  COP.  For  our  combat  net  radio  example,  only 
COPs  would  be  listed  as  key  parameters  in  the  APB. 

7.  The  IADDT  should  use  a  parameter  correlation  matrix  as  tool  to  ensure  that 
parameters  are  consistent  among  the  ORD,  TEMP  and  APB.  This  tool  also  established 
linkage  to  the  COEA.  The  matrix  also  shows  what  testing  is  planned  or  has  been 
executed  to  demonstrate  system  performance  for  each  parameter.  An  enclosure  to  the 
matrix  will  list  all  the  parameter  statements  with  thresholds  and  objectives.  This 
enclosure  will  also  provide  test  location  and  test  POCs.  An  example  parameter 
correlation  matrix  follows:  (See  figure  13) 


657 


Figure  13  (Sample  Parameter  Correlation  Matrix} 


FO# 

COT# 

MOE# 

COP# 

MOP# 

CP# 

TEST/DATE 

1 

1 

DT  4/3QFYXX 
IOTE/4QFYXX 

1-1 

1-1 

DT  4/3Q-FYXX 
IOTE/4QFYXX 

i-i-i 

1-1-1 

DT  4/3QFYXX 
IOTE/4QFYXX 

1-1-2 

1-1-2 

DT  4/3QFYXX 
IOTE/4QFYXX 

1-1-3 

1-1-3 

DT  4/3Q-FYXX 
IOTE/4QFYXX 

1-1-4 

1 

1 

2 

2 

DT5/2Q-FYXX 

IOTE/4QFYXX 

2-1 

2-1 

DT5/2Q-FYXX 

IOTE/4QFYXX 

2-1-1 

2-1-1 

DT5/2Q-FYXX 

IOTE/4QFYXX 

2-1-2 

2-2-2 

DT5/2Q-FYXX 

IOTE/4QFYXX 

As  a  final  example,  we  will  demonstrate  that  there  is  no  need  to  wait  for  the  "Brave  New 
World"  to  be  officially  implemented.  Most  of  the  recommended  procedures  can  be  implemented 
now  and  remain  within  DoD  5000  series  guidelines.  Applying  the  present  terminology  with  the 
proposed  numbering  system,  we  will  depict  one  COI  with  supporting/linked  parameters  for  the 
preceding  combat  net  radio  example.  We  will  specify  and  link  (as  much  as  possible)  the 
recommended  simplified  terminology  and  procedures  to  be  hopefully  approved  in  the  future. 
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Figure  14  (The  Situation  Today  with  Recommended  Procedures) 


The  Situation  Today 

Example  Performance  Parameter  Linkages  for  S INC  GARS 


IMOE/MAOPR  1-1)  IMOE  /  MAOPR  1-21 


1— 

} 

{CTP/MOP  1-2-3} 

Voice  message  completion  rates 
at  various  ranges  with  severe 
jamming  levels. 

Voice  message  completion  rates 
at  various  ranges  with 
moderate  jamming  levels. 

Voice  message  completion  rates 
at  various  ranges  with 
moderate  jamming  levels. 

Voice  message 
completion  rate, 
(no  jamming.) 


Voice  message 
completion  rate 
with  jamming. 


Now  we  will  repeat  the  process  using  the  recommended  simplified  terminology  with 
inherent  linkages. 


Figure  15  (The  Future  with  Recommended  Procedures  &  Simplified  Terminology) 


The  “Brave  New  World ” -The  Future 

Example  Performance  Parameter  Linkages  for  SINCGAR 


FO  1  {COE A}  &  COT  1{0RD,  TEMP } 


MOP  1-1-1{C0EA}  & 
CP  1-1-1  {TEMP,  ORD} 


MOP  1-1-2 {COE A}  & 
CP  1-1-2 {TEMP,  ORD} 


MOP  l-l-3{COEA}  & 
CP  1-1-3  {TEMP,  ORD} 
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IN  CONCLUSION 


We  have  discussed  how  to  specify  performance  parameters  and  establish  linkages  using 
the  present  performance  terminology.  This  process  is  challenging  but  possible  when  implemented 
by  an  integrated  acquisition  document  development  team  (IADDT).  The  two  preceding 
examples  illustrate  that  the  "Brave  New  World"  can  be  partially  implemented  today  and  still 
remain  within  the  DoD  5000  series  guidelines.  Why  wait?  Consistently  specified  "truly  critical" 
parameters  among  the  key  acquisition  documents  are  the  foundation  upon  which  a  major  defense 
acquisition  program  is  built.  Without  this  foundation,  the  program  will  fail  or,  even  worse, 
produce  a  weapon  system  that  is  fielded  and  does  not  meet  the  user's  requirements. 
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ENDNOTE  (Current  DoD  5000  Series  Definitions) 


1.  CRITICAL  OPERATIONAL  ISSUE  (COI)  --  A  key  operational  effectiveness  or 
operational  suitability  issue  that  must  be  examined  in  operational  test  and  evaluation  to 
determine  the  system’s  capability  to  perform  its  mission.  A  critical  operational  issue  is 
normally  phrased  as  a  question  to  be  answered  in  evaluating  a  system’s  operational 
effectiveness  and/or  operational  suitability.  {DoDI  5000.2,  Part  15} 

2.  CRITICAL  SYSTEM  CHARACTERISTIC  (CSC) 

a.  Critical  system  characteristics  are  those  design  features  that  determine  how  well 
the  proposed  concept  or  system  will  function  in  its  intended  operational 
environment.  CSCs  include  survivability;  transportability;  energy  efficiency;  and 
interoperability,  standardization,  and  compatibility  with  other  forces  and  systems 
including  support  infrastructure.  {DoDI  5000.2,  Part  4} 

b.  Critical  system  characteristics  will  address  electronics  counter-countermeasures 
(ECCM)  and  Wartime  Reserve  Modes  (WARM)  requirements;  conventional, 
initial  nuclear  weapons  effects,  and  nuclear,  biological,  and  chemical  contamination 
(NBC)  survivability;  natural  environmental  factors  (such  as  climatic,  terrain,  and 
oceanographic  factors);  and  electromagnetic  compatibility  and  frequency  spectrum 
assignment  for  systems  operating  in  the  electromagnetic  spectrum.  Define  the 
expected  mission  capability  (i.e.,  full,  percentage  degraded,  etc.)  in  the  various 
environments.  Include  applicable  safety  parameters  such  as  those  related  to  system, 
nuclear,  explosive,  and  flight  safety.  Identify  communications,  information,  and 
physical  and  operational  security  needs.  (DoD  5000. 2M,  Part  3-1 } 

3.  CRITICAL  TECHNICAL  PARAMETERS  (CTP)-  CTPs  are  derived  from  the 
Operational  Requirements  Document,  critical  system  characteristics  (see  Part  4  of 
DoD  Instruction  5000.2,  “Defense  Acquisition  Management  Policies  and 
Procedures”(reference  (b))  and  technical  performance  measures  (see  Section  6-A  of 
DoD  Instruction  5000.2,  “Defense  Acquisition  Management  Policies  and  Procedures” 
(reference  (b))  and  should  include  the  parameters  in  the  acquisition  program  baseline 
(see  Part  14  of  DoD  50002. M).  Discuss  the  relationship  between  the  critical  technical 
parameters  and  the  minimum  acceptable  operational  performance  requirements  in  the 
Operational  Requirements  Document.  (DoD  5000. 2M,  Part  7} 


4  .  FUNCTIONAL  OBJECTIVES  (FOs)  --  FOs  are  statements  describing,  in 

quantitative  terms,  the  tasks  a  system  will  be  expected  to  perform.  They  depend  upon 
the  type  of  system  at  issue.  For  example,  when  analyzing  transportation  systems,  the 
objectives  are  stated  in  terms  of  movement  requirements;  for  firepower  systems,  they 
reflect  the  types  of  targets  to  be  engaged.  The  effectiveness  of  system  alternatives  is 
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then  measured  in  terms  of  the  degree  to  which  the  functional  objectives  would  be 
attained.  {DoD  5000.2M,  Part  8} 


5.  KEY  PARAMETERS  -  Those  parameters  that  if  the  thresholds  are  not  met,  the 
milestone  decision  authority  would  require  a  reevaluation  of  alternative  concepts  or 
design  approaches.  {DoDI  5000.2,  Part  11-A} 

6.  MEASURE  OF  EFFECTIVENESS  (MOE)  -  MOEs  are  tools  that  assist  in 
discriminating  among  a  number  of  alternatives.  They  show  how  alternatives  compare 
in  meeting  functional  objectives  and  mission  needs. 1  Measures  of  effectiveness  should 
be  developed  to  a  level  of  specificity  such  that  a  system’s  effectiveness  during 
developmental  and  operational  testing  can  be  assessed  with  the  same  effectiveness 
criteria  as  used  in  the  COEA.  {DoD  5000. 2M,  Part  8} 

7.  MEASURE  OF  PERFORMANCE  (MOP)  -  MOPs  are  not  specifically  defined  by 
DoD  instruction  or  manual;  however,  a  commonly  accepted  definition  follows:  MOP 
is  defined  as  a  quantitative  measure  of  the  lowest  level  of  physical  performance  (e.g. 
range,  velocity,  throughput)  or  physical  characteristic  (e.g.,  height,  weight,  volume, 
frequency)  of  a  system.  MOPs  are  derived  from  MOEs.  (AFMAN  10-603} 

8  MINIMUM  ACCEPTABLE  OPERATION  PERFORMANCE 

REQUIREMENTS  (MAOPRsf  -  A  summary  of  the  critical  operational 
effectiveness  and  suitability  parameters  and  constraints  (manpower,  personnel, 
training,  software,  computer  resources,  transportation,  and  etc.)  described  in  the 
Operational  Requirements  Document  (ORD).  {DoD  5000 . 2M,  Part  7} 


9.  OBJECTIVE-  A  measurable,  beneficial  increase  in  capability  or  operations  support 
above  the  threshold.  (DoD  5000.2M,  Part  3-1} 

10.  PARAMETER  -  A  determining  factor  or  characteristic.  Usually  related  to 
performance  in  developing  a  system.  (Commonly  accepted  definition-Not  specifically 
defined  in  DoD  5000  series} 


11.  PERFORMANCE-  Those  operational  and  support  characteristics  of  the  system  that 
allow  it  to  perform  effectively  and  efficiently  its  assigned  mission  over  time.  (DoDI 
5000.2,  Part  15} 

12.  REQUIREMENT-  Is  a  mission  need.  Requirements  are  initially  stated  in  broad 
operational  terms  and  are  progressively  refined  in  both  number  and  specificity  during 
each  acquisition  phase.{DoDD  5000.1,  Part  2} 


663 


13.  TECHNICAL  PERFORMANCE  MEASURES  (TPM)  -  Performance  measures 
that  are  developed  and  maintained  throughout  the  process.  These  measures  will  be 
used  to  assess  how  well  the  evolving  design  meets  the  system  requirements.  {DoDI 
5000.2,  Part  6-A} 

14.  THRESHOLD  -  The  minimum  acceptable  value  required  to  satisfy  the  mission 
need.  {DoDI  5000.2,  Part  4-B} 
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MANAGING  FLIGHT  TEST  PROGRAMS: 
An  Earned  Value  Approach 

Perry  L.  Lamy,  Colonel,  USAF 
Air  War  College 


ABSTRACT 

Traditional  measures  of  flight  test  progress, 
such  as,  counting  flight  hours  or  completed 
test  points,  are  limited  in  deciding  resource 
allocation  and  prioritizing  test  activity.  A 
systems  engineering  approach  was  utilized  to 
improve  the  planning,  controlling,  and  status 
reporting  of  the  flight  test  program.  Earned 
value,  in  terms  of  test-point  hours,  provides 
a  normalized  measure  of  work  effort  (test 
activity).  This  concept  was  derived  from  the 
DOD  Cost/Schedule  Control  System  Criteria 
and  employed  with  success  in  a  large  aircraft 
developmental  flight  test  program. 

INTRODUCTION 

You  can 7  control  what  you  can  7  measure. 1 

— Tom  DeMarco 

Among  the  many  challenges  encountered  in 
the  management  of  major  weapon  system 
acquisition  programs  is  the  balance  between 
system  cost,  system  schedule,  and  system 
effectiveness.2  A  good  project  manager 
must  consistently  be  able  to  focus  the  “right 
resources”  on  the  “right  problem”  at  the 
“right  time”  to  obtain  acceptable  results. 
Each  phase  of  defense  acquisition  presents 
the  program  manager  with  the  challenge  of 
integrating  and  coordinating  a  set  of  work 
processes  while  controlling  the  risk  to  cost, 
schedule,  and  effectiveness.  Large,  high 
visibility  defense  acquisition  programs 
present  unique  challenges.  The  “high 
visibility”  creates  an  abundance  of  scrutiny 
and  oversight.  The  “large”  means  more 


resources,  more  complex  work  processes, 
and  therefore,  more  risk.3 

Flight  testing  air  vehicles  is  one  of  the  work 
processes  found  in  aircraft  acquisition 
programs  and  is  accompanied  by  similar 
challenges  in  controlling  cost,  schedule,  and 
effectiveness.  In  large,  high  visibility  aircraft 
development  programs,  the  flight  test  project 
can  become  a  program  within  a  program. 
The  test  manager’s  task  is  to  convert  test 
resources  into  output  using  test  processes  to 
achieve  a  program  objective.4 

Traditionally,  the  flight  test  community  has 
focused  on  the  technical  aspects  of  flight  test 
(planning  the  test,  conducting  the  test,  and 
reporting  the  results)  and  had  a  weak 
commitment  to  the  problems  of  the  program 
office  in  terms  of  cost  and  schedule.  “That’s 
the  SPO’s  problem!”5  The  tester  saw  the 
priority  as  delivering  the  most  capability  to 
the  user  and  identifying  system  deficiencies, 
regardless  of  the  impact  of  this  activity  on 
cost  and  schedule.  Consequently,  the 
program  manager  was  often  surprised  by 
problems  encountered  in  flight  test,  then  was 
left  alone  to  recover  within  program  cost  and 
schedule  constraints.  Overruns  in  flight  test 
commonly  led  to  overruns  in  program  cost 
and  schedule  or  delivery  of  less  capability  to 
the  user.6 

The  situation  has  changed.  Cutbacks  in 
defense  programs  and  test  budgets,  previous 
failures  in  acquisition  programs,  and  the 
increase  oversight  of  DOD  and  Congress 
require  greater  efficiency  and  better  program 
management.  To  improve  the  weapon 
system  acquisition  process.  Air  Force 
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Material  Command  (AFMC)  introduced  a 
variety  of  programs,  such  as.  Integrated 
Product  Development,  Integrated  Weapon 
System  Management,  and  Single  Face  to  the 
Customer.  In  addition,  the  Acquisition 
Professional  Development  Program  and  a 
professional  acquisition  corps  will  result  in 
more  effective  and  efficient  management.7 

The  incorporation  of  quality  concepts  and 
the  introduction  of  integrated  product  teams8 
mean  testers  and  program  managers  must 
become  better  partners  for  the  success  of  the 
program.  Test  managers  who  may  believe 
that  flight  test  is  “unmanageable”  must  now 
assume  a  stake  in  the  responsibility  of 
program  management. 

Are  large  flight  test  programs  manageable ? 
Despite  their  scientific  approach  and 
technical  competence,  flight  testers  often 
lack  the  knowledge  of  or  training  in  program 
management  tools  to  control  cost  and 
schedule.  Large  programs  are  manageable 
but  traditional  test  management  tools  are 
inadequate. 

The  purpose  of  this  paper  is  to  describe  a 
successful  innovation  in  the  risk  management 
of  a  large  developmental  flight  test  program. 
By  following  a  systems  approach  and 
modifying  several  program  management 
tools  and  methods,  the  quality  of  the  test 
process  was  significantly  improved.  This 
change  was  introduced  almost  four  years  ago 
and  substantially  enhanced  the  planning, 
controlling,  and  progress  reporting  of  the 
major  weapon  system  program.9 

BACKGROUND 

Thing ’s  could  be  worse. 

— Princess  Leia  Organa 

They  ’ re  worse ! 10 

— Han  Solo 


Several  reasons  demand  concern  for 
improving  the  management  of  large  flight 
test  programs.  First,  overruns  in  cost  and 
schedule  are  typical  for  defense  acquisition 
programs.  A  look  at  24  defense  acquisition 
programs  showed  an  average  cost  overrun  of 
45  percent  and  a  schedule  overrun  of  65 
percent  in  the  Engineering  and 
Manufacturing  Development  phase  of  the 
programs.11  Second,  flight  testing  is  an 
expensive  process.  The  government  cost  for 
flight  test  of  a  large  aircraft  development 
program  can  average  $750,000  to  $1  million 
each  week.12  Next,  the  new  acquisition 
directives  and  regulations  that  resulted  from 
DOD  acquisition  reform,  most  notably  the 
DODD  5000  and  the  AF  Instruction  99 
series,  require  a  more  disciplined  approach  to 
the  conduct  of  program  acquisition  and  flight 
test  project  management.  Finally,  the  quality 
movement  and  the  emphasis  on  quality 
practices  within  AFMC,  have  motivated  an 
endless  drive  for  continuous  improvement  of 
all  acquisition  processes. 

The  spiraling  cost  of  aircraft  weapons  in  an 
environment  of  decreasing  defense  budgets 
and  the  need  to  be  “leaner  and  meaner,” 
means  that  time  spent  in  flight  test  must  be 
leveraged  to  maximize  the  return  on  every 
acquisition  dollar  spent  in  obtaining 
capability  for  the  user,  efficiently,  and  within 
schedule.  Appreciating  the  need  for  an 
improved  flight  test  management  system 
requires  a  look  at  the  challenges  of  program 
management,  the  competing  nature  between 
program  management  and  flight  test,  and  the 
test  process. 

Project  Model 

Both  program  acquisition  and  its  subset,  the 
flight  test  program,  can  be  viewed  in  terms 
of  the  simple  model  depicted  in  Figure  1. 
Both  processes  involve  converting  resources 
into  output  using  work  processes.  The 
output  satisfies  an  objective  expressed  as 


666 


meeting  some  standard  or  expectation  in 
terms  of  cost,  schedule,  and  effectiveness. 
Generally,  the  user  or  customer  specifies  the 
objective.  Unknowns  will  exhibit  themselves 
as  risk  to  cost,  schedule,  and  effectiveness. 
Risk  is  the  probability  of  an  undesirable 
event  occurring  and  the  impact  of  the  event 
on  project  cost,  schedule,  and  effectiveness. 
Minimizing  risk  decreases  the  “probability” 
of  the  event  or  its  “impact.”  Risk  is 
minimized  using  a  set  of  management  tools 
to  control  the  work  processes.13 

Flight  test  work  processes,  such  as,  test 
planning,  technical  and  safety  reviews, 
engineering,  configuration  control,  and  data 
analysis,  are  used  by  the  flight  test  manager 
to  produce  output  in  the  form  of  test  data 
results.  Technical  risk  is  inherent  due  to 
design  problems,  failure  of  hardware  or 
software,  and  other  deficiencies  discovered 
during  “testing  that  decrease  system 
effectiveness.  Management  risk  results  from 
making  decisions  to  deal  with  technical 
problems  and  taking  actions  to  meet  cost  and 
schedule  constraints.  Risk  encountered  in 
flight  test  can  cause  slips  in  the  program 
acquisition  schedule  and  additional  cost. 


Testing  in  Systems  Acquisition 

A  test  is  a  disciplined  process  that  follows 
the  scientific  method  of  objective, 
hypothesis,  experiment,  data  collection, 
analysis,  and  results.  By  definition  a  test 
must  be  repeatable,  otherwise  it  is  not  a  valid 
test.  The  same  initial  conditions  and  the 
same  test  procedures,  will  result  in  the  same 
conclusions.  Tests  validate  technical 
specifications  and  identify  capabilities, 
limitations,  and  deficiencies.  The  purpose  of 
testing  as  part  of  systems  acquisition  is  to  (1) 
provide  timely,  accurate,  and  affordable 
information  to  the  program  manager  for 
assessing  military  utility  and  cost 
effectiveness;  (2)  reduce  risks  associated 
with  acquisition;  and,  (3)  ensure  the 
acquisition  community  is  delivering 
operationally  effective  and  suitable  systems 
to  the  Air  Force.  The  end  result  of  testing  is 
the  delivery  of  weapon  system  capability  and 
the  fulfillment  of  a  specific  objective.14 

The  “fly-fix-fly”  test  methodology  prevalent 
during  the  last  decade  led  to  a  short-term 
management  view  in  flight  test, 
concentrating  on  the  immediate  problem  and 
losing  sight  of  the  bigger  picture.  To  correct 
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this  deficiency,  the  test  process  was 
redefined.  AFI  99-103  replaced  the  short¬ 
term  viewpoint  of  “fly-fix-fly”  with  a 
discipline  scientific  approach  of  the  “predict- 
test-compare”  methodology.  It  is  the  goal 
of  this  process  to  produce  effective  and 
suitable  systems  that  satisfy  the  user’s  needs 
by  reducing  the  risk  to  cost,  schedule,  or 
effectiveness.15 

The  objectives  of  a  flight  test  program  are 
specified  in  a  test  plan.  The  test  team 
prepares  this  document  which  details  the 
specific  questions  that  will  be  answered 
(objectives);  the  anticipated  or  required 
performance  (hypothesis);  the  test  conditions 
and  procedures  used  to  acquire  the  data 
(experiment);  and,  the  data  required  and  the 
evaluation  technique  (analysis).  The  time 
spent  in  flight  (flight  hours)  becomes  the 
resource  that  must  be  prudently  managed,  as 
this  is  the  only  opportunity  data  can  be 
collected  to  fulfill  the  requirements  of  the 
test  plan.  In  effect,  the  flight  test  manager’s 
task  is  to  efficiently  and  to  effectively 
transform  flight  test  hours  into  time  “on  test 
condition,”  which  is  the  time  spent  collecting 
data. 

The  changes  to  a  more  disciplined  test 
process  demand  the  tester  take  more 
responsibility  to  complete  testing  utilizing 
the  resources  and  within  the  schedule 
specified.  If  this  is  not  possible,  the  program 
manager  must  be  informed  of  a  potential 
breach  in  the  baseline  in  sufficient  time  to 
minimize  impact  on  program. 

Flight  Test  Dilemma 

The  challenge  for  both  the  program  and  test 
managers  is  to  identify  and  minimize  all 
potential  risks  to  effectiveness  within  a 
specified  budget  and  schedule.  The  three 
factors-cost,  schedule,  and  effectiveness-are 
interdependent.  In  the  event  of  unanticipated 
problems,  delivery  of  specified  effectiveness 
will  increase  cost  and  schedule.  If  either  cost 


or  schedule  is  constrained,  then  the 
equilibrium  may  only  be  regained  by  a 
sacrifice  in  effectiveness.  The  typical 
experience  in  acquisition  finds  continual 
pressure  against  cost  and  schedule  to  deliver 
the  required  effectiveness. 

For  the  program  manager  trying  to  keep  cost 
and  schedule  under  control,  the  testing 
activity  can  only  bring  bad  news.  If 
everything  goes  right,  then  there  is  no  impact 
to  cost  and  schedule.  But  this  rarely 
happens.  Most  likely  problems  will  be 
discovered,  delays  will  result,  and  both  cost 
and  schedule  will  increase.  This  presents  the 
tester  with  a  competing  set  of  commitments: 
the  first  to  the  program  manager  to  reduce 
risks  and  make  schedule;  the  second,  to  the 
user  to  provide  fully  tested  equipment  with 
the  desired  capability.  Satisfying  both  of 
these  commitments  becomes  the  flight  test 
dilemma. 

Projects  fall  behind  for  several  reasons:  poor 
estimates  of  the  work  effort,  lack  of 
understanding  of  requirements,  inadequate 
risk  assessment  or  unanticipated  problems 
(unknown-unknowns).  Projects  can  also  get 
into  trouble  through  poor  management 
practices:  pressure  to  deliver,  poor 

monitoring  of  progress,  confusing  effort  with 
progress,  or  adding  more  resources  to  a 
project  already  behind.  Although  little  can 
be  done  about  unknown  problems,  the 
greatest  leverage  against  cost,  schedule,  and 
effectiveness  risks  is  found  in  good 
management  actions.  This  requires  a 
visionary  view  over  the  entire  course  of  the 
flight  test  program;  appreciating  the  context 
of  the  acquisition  program  and  its 
milestones;  and,  adopting  a  risk  management 
approach. 

Nature  of  the  Problem 

There  are  three  aspects  to  the  long  range 
management  challenge  of  flight  testing: 
planning,  controlling,  and  reporting  progress. 
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HOW  DO  WE  MEASURE  FLIGHT  TEST  PROGRESS? 

TRADITIONAL  “MEASURES”. . . 


Figure  2:  Traditional  Measures  of  Flight  Test  Progress 


Each  area  needs  management  attention  to 
control  risk  during  the  flight  test  process. 

Detailed  long-range  planning  of  complex 
programs  is  labor  intensive  and  time 
consuming.  The  inherent  uncertainty  of 
development  programs  complicates  a  logical 
approach.  Previously,  flight  test  planning 
was  limited  to  test  objectives,  test 
procedures,  test  conduct,  and  technical  and 
safety  concerns.16  Today,  program  concerns 
are  equally  important.  The  program 
manager  will  ask  specific  questions  of  the 
tester:  Is  there  enough  time  to  satisfy  all 
obligations?  How  long  will  a  specific  test 
take?  When  will  a  modification  be  delivered 
and  what  are  the  prerequisites  required?  In 
addition,  every  decision  in  a  complex 
program  will  come  with  both  liabilities  and 
benefits.  Choosing  the  right  alternative  at  a 
decision  point  requires  the  knowledge  of  its 
impact  on  cost  and  schedule.  Long-range, 
end-to-end  planning  and  the  integration  of 
test  activity  are  now  possible  due  to 


powerful  personal  computers  and  project 
manager  software.  Long-range  program 
planning,  properly  performed,  is  invaluable  in 
helping  with  decision  making  and  providing 
answers. 

During  execution,  controlling  the  course  of 
the  test  project  toward  the  objectives  will 
depend  on  reliable,  accurate  information  and 
the  ability  to  assess  the  impact  of  an  event  on 
the  project’s  future.  Good  estimates  and  a 
detailed  plan  will  help  in  decision  making  if 
the  planning  tool  can  provide  timely 
forecasts  and  results.  Typical  questions  will 
include:  What  is  the  impact  of  a  failure — the 
incorrect  design  of  a  system  or  the  loss  of  an 
airplane — on  the  schedule?  What  is  the 
effect  of  cutting  3  months  out  of  the 
schedule  or  adding  $10  million  dollars  to  the 
project?  Is  there  enough  time  to  complete  all 
requirements  at  the  current  flight  rate? 
Smart  control  requires  accurate  forecast 
based  on  quantifiable  and  defendable  work 
process  metrics. 
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Likewise,  reporting  status  of  large,  high 
visibility  programs  is  not  straightforward. 
These  programs  are  characterized  by 
complex  systems,  long  duration,  and  a  large 
test  matrix.  Outsiders,  often  program 
managers  without  an  appreciation  for  the 
detail,  want  a  simple  answer  to  the  simple 
question:  Are  we  on  schedule?  Of  course, 
this  question  spawns  an  entire  set  of  follow- 
on  queries:  Will  we  finish  on  time?  What  is 
the  impact  of  this  problem  on  schedule?  Can 
the  schedule  be  made  up?  What  will 
additional  resources  provide  in  schedule? 
Congress  is  even  less  tolerant — How  much 
testing  is  complete?  Providing  a  straight 
forward  answer  of  “63  %”  may 
communicate  an  incorrect  conclusion:  Has 
63%  of  effort,  or  63%  of  funds,  or  63%  of 
schedule  been  expended?  Each  is  a  different 
viewpoint. 

Traditional  methods  of  measuring  progress 
in  the  flight  test  program  include  counting 
(1)  completed  test  points,  (2)  flight  hours 
flown,  (3)  objectives  completed,  or  (4)  a 
combination  of  these  measures.  For 
example,  if  a  program  contains  1000  test 
points  and  400  have  been  completed,  then 
the  answer,  “40%  of  the  test  points  are 
complete”  might  be  interpreted  as  “the 
program  is  40%  complete.”  Likewise,  a 
100-flight-hour  program  with  30  hours  flown 
may  be  30%  complete  rather  than  “30%  of 
the  flight  hours  have  been  expended”;  or,  5 
of  8  objectives  are  completed  and 
consequently  “62%  complete”.  Each 
measure  suffers  from  the  same  shortcoming. 
They  convey  only  information  regarding  the 
specific  measures — test  points,  flight  hours, 
or  objectives — but  are  often  erroneously 
taken  as  an  indication  of  the  work 
performed.  Although  useful  to  show  the 
status  of  a  resource,  they  are  incomplete  in 
predicting  future  status,  progress,  or  work 
completed.  Ideally  what  is  needed  is  one 


measure  that  could  normalize  the  work  by 
equating  effort,  cost,  and  schedule. 

These  three  aspects  of  project  management¬ 
planning,  controlling,  and  reporting-are  not 
unique  to  the  effort  of  flight  test  as  each  is 
found  in  every  project  management  situation. 
Unfortunately,  a  community  founded  in  the 
scientific  method  more  often  resorts  to 
insufficient  management  planning  practices 
rather  than  a  disciplined,  systems  analysis 
approach  of  “planning  the  work  and 
working  the  plan”  using  metrics  and 
meaningful  measures  of  progress. 

The  Requirements  for  Improvement 

Given  a  perspective  of  the  purpose  of 
testing,  the  flight  test  dilemma,  and  the 
inherent  problems  of  traditional  approaches 
to  flight  test  management,  the  requirements 
of  a  risk  management  system  to  counter  the 
shortfalls  caused  by  the  problems  identified 
can  be  stated.  Specifically,  the  system 
should: 

•  assist  in  planning,  controlling,  and 
reporting  of  the  flight  test  project; 

•  measure  progress  towards  individual 
test  objectives  and  program 
milestones; 

•  provide  an  insight  into  flight  hour 
requirements  to  meet  major  test 
program  milestones; 

•  measure  and  track  the  use  of  flight 
test  resources;  and, 

•  identify  critical  paths,  resource 
availability/constraints,  test  force 
capacity,  and  program  milestones. 

To  meet  these  conditions,  the  solution  had  to 
fulfill  additional  needs.  To  assist  in  planning 
and  integrate  test  activities,  prerequisites, 
milestones,  and  resources,  use  of  PC-based 
project  management  software  is  required. 
The  software  should  accommodate  the 
specification  of  exit  criteria;  the  identification 
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of  critical  paths,  critical  resources,  or 
duplication  of  effort;  and,  support  flight  test 
and  mission  planning.  For  the  control 
function,  the  system  should  be  capable  of 
accepting  changes  in  the  plan,  along  with 
providing  a  means  of  assessing  impact  from 
“unknown-unknowns,”  changing  require¬ 
ments,  or  “what-if’  type  questions  in  a 
timely  manner.  Finally,  the  system  should  be 
able  to  track  and  report  progress  using  a 
normalized  measure  of  work,  support  test 
personnel  and  project  decision  makers,  and 
produce  reports  for  use  by  senior  program 
management  officials  to  understand  or 
convey  progress  of  the  test  program. 

A  NEW  PARADIGM 

Do  not  follow  where  the  path  may  lead.  Go 
instead  where  there  is  no  path  and  leave  a 
trail.  17 

— Anonymous 

Development  of  a  risk  management  process 
followed  a  system  analysis  approach.  The 


concept  adopted  was  based  on  systems 
engineering  and  the  contract  management 
tool  used  by  DOD.18  To  measure  the  work 
effort,  an  earned  value  basis  was  used  which 
allows  for  realistic  cost  and  schedule 
assessment.  The  development  process 
consisted  of  5  steps  as  shown  in  Table  1. 

Step  1:  Identify  Requirements 

Three  types  of  requirements  influence  flight 
test  management:  test  requirements, 

program  milestones,  and  resources  required. 
Test  requirements  evolve  from  user  opera¬ 
tional  needs,  contractual  specifications,  and 
exit  criteria  for  milestone  decisions.  The 
user’s  operational  needs  are  translated  into 
technical  requirements.  Specifications  are 
the  technical  requirements  for  which  the 
manufacturer  is  contractually  committed. 
Technical  requirements  that  are  quantifiable 
and  testable  become  test  requirements. 
Completion  of  a  test  requirement  requires  a 
specific  set  of  tests  to  be  performed.  Data 


Table  1:  System  Engineering  Approach  to  Managing  Flight  Tests 


1  Identify  requirements 


2  Define  the  work 


3  Schedule  the  work 


Baseline  the  work 


Execute  the  plan 


Outcome 


Test  requirements 
Program  milestone 
Resources  required 


Estimate  of  work  required  to 
accomplish  each  test  requirement 
Work  sequence  (network) 

Work  packages 


Work  path  constrained  by 
milestone  and  resources 
Time-phased  work  plan 


Budget  at  completion 
Schedule  of  work  effort  overtime 
Commitment  to  plan 


Metrics  to  monitor  performance 
against  plan 
Adjustments  to  plan 
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Figure  3:  Product  Work  Breakdown  Structure  for  an  Aircraft 


obtained  from  these  tests  are  used  to  show 
achievement  or  failure  of  the  test  require¬ 
ment.  The  outcome  of  this  step  is  achieved 
when  the  test  to  accomplish  each 
requirement  is  identified  and  prioritized. 

Program  milestones  are  identified  in  program 
documents,  such  as.  Acquisition  Program 
Baseline,  Program  Management  Directive, 
Test  and  Evaluation  Master  Plan,  or  a  master 
schedule. 19  These  are  program  commitments 
that  the  test  program  must  support. 
Examples  include  production  decisions, 
interim  reports,  end  of  testing,  and  reports  to 
Congress.  All  major  program  milestones 
affected  by  test  activity  must  be  identified  in 
this  step. 

Finally,  the  availability  of  resources 
necessary  to  accomplish  testing  must  be 
determined.  This  includes  the  delivery  and 
configuration  of  test  articles,  special  test 
assets  or  facilities,  and  personnel. 
Requirements'  identification  will  be  a 
continuous  process  as  changes  will  occur 
during  the  conduct  of  testing.  All  changes  or 
added  requirements  must  be  critically 
evaluated  prior  to  incorporation  in  the 
schedule  and  commitment  of  resources. 


Step  2:  Define  the  Work 

Test  requirements  specify  the  work  that  must 
be  performed.  This  step  quantifies  the  work 
and  charts  the  work  path.  The  specific  test 
work  is  described  in  the  test  plan.  Test  plans 
translate  requirements  into  test  points.  Test 
plans  are  normally  functionally  oriented  by 
engineering  discipline  or  aircraft  system. 
The  test  plan  contains  a  test  objective,  the 
initial  conditions  of  the  test,  the  procedure  to 
be  followed,  and  the  data  needed  to  satisfy 
the  test  objective.  Large  development  flight 
test  programs  can  have  dozens  of  test  plans, 
therefore,  a  Work  Breakdown  Structure 
(WBS)  is  used  to  breakdown  and  organize 
this  effort. 

The  WBS  is  a  system  engineering  tool  that 
organizes  and  defines  the  work  to  be 
performed  in  functional  oriented  or  product 
oriented  categories.20  The  WBS  is  used  to 
break  complex  systems  into  subsystem  and 
component  levels.  An  example  WBS  for  an 
airplane  and  expanded  along  the  avionics 
system  is  shown  in  Figure  3.  The  complete 
WBS  would  expand  each  box  down  to  a 
component  level.  At  this  level  or  tier,  the 
work  effort  required  to  produce  the 
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Figure  4:  WBSfor  Flight  Test  Program 


component  can  be  specifically  defined  and  is 
called  a  work  package.  By  summing  up  each 
tier,  the  work  effort  required  to  produce  any 
sub-system  or  the  total  aircraft  can  be 
obtained. 

A  function  oriented  WBS  is  used  to  organize 
all  the  test  work  by  engineering  discipline 
down  to  the  work  package  level.  A  flight 
test  work  package  is  a  functional  breakdown 
of  test  points,  that  are  similar  in  procedure 
and  configuration.  Each  work  package 
contains  specific  test  points  identified  by  a 
unique  number  and  the  expected  cost  of 
completing  this  work.  The  expected  or 
budgeted  cost  of  work  is  measured  in  test- 
point  hours. 

A  flight  hour  is  broken  into  test-point  hours 
and  overhead.  Overhead  is  a  fact-of-life  cost 
of  operation.  It  includes  takeoff,  cruise  to 
the  test  range,  air  refueling  (to  extend  flight 
time),  configuration  changes,  repeated  test 
points,  problems  with  the  aircraft  or 
instrumentation,  and  landing.  Normally, 


during  these  activities  no  productive  testing 
or  work  is  accomplished.  The  test-point 
hour  is  the  time  spent  acquiring  data  for  a 
test,  the  “on-condition”  time.  The  overall 
cost  of  flight  testing  is  measured  in  flight 
hours  and  work  complete  is  measured  by 
test-point  hours.  The  ratio  between  the  two, 
the  efficiency  of  converting  flight  hours  into 
test-point  hours,  is  called  the  flight  test 
efficiency.  The  composition  of  a  flight  hour 
and  overhead  is  shown  in  Figure  5.  The  use 
of  test-point  hours  allowed  each  point  to  be 
normalized,  thus  allowing  a  direct  work 
effort  comparison  between  every  test  point. 

Finally,  the  tests  required  to  achieve  each 
test  objective  is  defined.  The  path  through 
the  test  plans  is  specified  and  the  sequence  of 
testing  is  defined.  These  paths  are  assembled 
into  a  network  representing  the  complete  test 
effort.  Figure  4  shows  an  example  PERT 
network21  of  activity.  Boxes  identify  test 
points  or  work  to  be  accomplished  and 
rounded  boxes  are  test  requirements: 
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Figure  5:  Anatomy  of  a  Flight  Test  Hour 


program  milestones,  test  objectives,  or 
constraints.  This  is  a  very  labor  intense  part 
of  the  process.  Each  functional  discipline 
must  be  integrated  into  the  network. 
Undefined  test  activity  or  work  must  be 
accounted  for  with  best  estimates  until 


further  detail  is  available. 

Step  3:  Schedule  the  Work 

In  this  step,  the  output  of  Step  2,  the  work 
activity  network,  is  constrained  by  test 
milestones  and  available  resources.  The 
PERT  network  is  transformed  into  a 


Tasks  with  Test  Test  Objective 

Hours  Specified  ^ —  Test  Milestones 


Figure  6:  Defining  and  Scheduling  the  Work 


V 
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schedule  by  adding  dates.  The  outcome  of 
this  sten  is  a  time-nhased  nlan  that  soecifies 
the  work  effort  within  a  defined  schedule,  as 
shown  in  Figure  6. 

The  anticipated  fly  rate  will  determine  the 
capacity  to  expend  flight  test  hours.  The 
availability  of  the  aircraft  and  associated  test 
support  resources  will  determine  how  often 
it  can  fly  and  what  fly  rate  (flight 
hours/month)  can  be  obtained.  Using  flight 
test  efficiency  and  converting  flight  hours 
into  test-point  hours,  the  amount  of  schedule 
required  to  accomplish  the  entire  test 
program  can  be  estimated.  Constraints  will 
create  “critical  paths”  through  the  network. 
Shortfalls  are  identified  and  management 
action  can  be  taken  to  alleviate  the 
scheduling  pressure.  The  outcome  of  this 
step  should  be  a  schedule  that  management 
can  support. 


Step  4:  Baseline  the  Work 


Once  a  schedule  constrained  plan  is  defined, 
the  time-phased  accomplishment  of  work  can 
be  determined.  This  involves  the  summation 
of  each  flight  test  work  package  as  a 
function  of  time.  The  result  is  a  “baseline” 
that  equates  work  effort  over  calendar  time. 
The  baseline  is  an  invaluable  tool  as  it  will  be 
used  during  execution  of  the  test  program  to 
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Figure  7:  Flight  Test  Program  Management 
Baseline 


track  progress  and  determine  impact  of 
changes  on  the  overall  schedule.  The 
baseline  can  be  expressed  in  terms  of  work 
effort  (test-point  hours)  or  cost  (flight 
hours).  As  previously  described  the  ratio 
between  the  two  is  flight  test  efficiency  that 
is  initially  assumed  for  planning  but  is 
measured  once  testing  begins.  The  “cost” 
of  the  entire  flight  test  effort  can  be 
estimated  and  “budget  at  completion”  is 
used  to  control  the  flight  test  effort.  Figure  7 
shows  an  example  program  baseline  in  terms 
of  cost  for  a  2,700  flight  hour  test  program. 

Step  5:  Execute  the  Plan 

Following  coordination  of  the  flight  test 
baseline  with  the  program  manager, 
execution  of  the  schedule  begins.  The  flight 
test  manager  must  now  inform  the  program 
office  when  a  breach  in  the  baseline  occurs. 
A  breach  is  an  anticipated  overrun  in  cost  or 
schedule  by  a  predetermined  amount.  Action 
to  resolve  the  breach  may  require  additional 
resources,  management  attention,  and  may 
even  involve  a  re-baselining  of  activity. 

As  work  packages  are  completed,  credit  for 
the  value  of  the  work  is  taken  using  the  test- 
point  hours  that  make  up  the  work  package. 
This  is  the  earned  value  aspect  of  the  system. 
Despite  the  cost  in  flight  hours,  only  the  test- 
point  hours  budgeted  for  that  specific  work 
effort  measure  the  value  added  by  flight  test. 

Use  of  earned  value  to  measure  the  work 
effort  provides  a  more  complete  indication  of 
flight  test  status.  For  example,  if  2,000  test- 
point  hours  of  work  define  the  program, 
accomplishing  500  test-point  hours  means 
25%  of  the  flight  test  work  is  complete.  This 
is  a  much  improved  measure  of  progress 
when  compared  to  counting  flight  test  hours 
or  test  points  completed.  During  execution, 
a  set  of  flight  test  metrics  is  generated  to 
chart  progress.  Example  metrics  are  shown 
in  Figure  8. 22  These  metrics  can  be  used  by 
all  levels  of  management  to  determine  status 
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Test-Point  Hours  Per  Missions 


Figure  8:  Examples  of  Flight  Test  Metrics 


and  in  decision  making.  In  addition,  these 
same  metrics  can  be  used  to  brief  agencies 
with  oversight  authority  on  progress  of  the 
program.  The  flight  test  manager  can 
monitor  performance  against  the  baseline  and 
make  better  informed  decisions.  The 
addition  or  deletion  of  test  requirements  can 
be  evaluated  against  effects  on  the  entire 
flight  test  program  or  intermediate 
milestones. 

Implementation 

Obviously  in  a  large  program  such  a  system 
cannot  be  performed  by  hand  because 
variables  and  interactions  number  in  the 
thousands.  Use  of  PC  based  project 
management  software  is  a  necessity.  The 
project  management  software  should  allow 
the  construction  of  WBS  and  networks, 
specifying  priorities  and  resource  allocation, 
and  generating  reports  and  metrics.  These 


systems  typically  measure  cost  in  “dollars” 
but  test-point  hours  can  be  substituted. 

In  addition,  the  flight  test  team  will  need 
training  in  the  concepts  of  earned  value, 
WBS,  and  systems  engineering  concepts. 
Training  should  include  how  to  use  and 
interpret  the  metrics  generated  to  determine 
status  and  schedule  of  tests. 

RESULTS 

Don  7  try  to  innovate  for  the  future. 
Innovate  for  the  present!  23 

— Peter  F.  Drucker 

In  1991,  the  B-2  flight  test  program  at 
Edwards  APB  faced  a  significant  increase  in 
complexity  due  to  continued  high  visibility 
and  oversight  from  DOD  and  Congress,  an 
ever  expanding  test  matrix,  receipt  of 
additional  test  vehicles  (each  different  in 
configuration  and  purpose),  and  the  need  to 
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fly  each  aircraft  more  than  once  a  week.  The 
program  suffered  from  the  same  malady 
afflicting  other  flight  test  programs-a  short 
term  view  and  the  inability  to  project  further 
than  several  weeks  or  months  into  the  future. 
The  “planning  system”  in  place  would  not 
adapt  to  an  increase  in  complexity.  The 
program  was  composed  of  15,000  test 
points,  3,600  flight  hours,  6  flight  test 
vehicles,  and  a  test  team  of  almost  2,000 
people.  A  project  management  system  was 
needed  to  efficiently  use  flight  test  hours, 
meet  schedule,  predict  shortfalls  in  cost  or 
schedule,  and  report  progress. 

The  flight  test  team  set  out  to  develop  a  risk 
management  system  with  a  compatible  set  of 
planning  tools  capable  of  accommodating  the 
increase  in  program  complexity.  The  system 
approach  described  in  this  paper  was  used. 

There  were  many  obstacles  along  the  way. 
The  flight  test  culture  inbred  at  Edwards 
AFB  took  several  months  to  overcome. 
Many  did  not  understand  the  principles 
fundamental  to  the  system.  The  metrics 
initially  devised  to  plan,  control,  and  report 
progress  were  inadequate  and  required 
continuous  refinement.  Finally,  the  logistics 
and  ground  test  activity  were  not  modeled  or 
incorporated  into  the  flight  test  model.  This 
has  become  the  biggest  risk  to  schedule  as 
aircraft  downtime  and  the  maintenance  work 
that  needs  to  be  performed  are  continually 
underestimated.  These  areas  will  require 
additional  work  and  ideas. 

It  has  been  almost  four  years  since  this 
system  was  developed.  During  this  period 
additional  transformations  in  the  flight  test 
program  have  taken  place.  Each  was 
necessary  due  to  the  evolution  in  the 
program  along  with  real  world  developments 
unforeseen  in  1991.  Integrated  Product 
Teams  (IPTs)  were  incorporated  (see  Figure 
4)  and  empowered  with  a  budget  of  test- 
point  hours  to  conduct  their  tests.  The  IPTs, 


using  metrics  from  the  management  model, 
are  now  accountable  and  must  manage 
within  budget  or  project  overruns.  The  flight 
test  model  is  used  to  integrate  between  the 
functional  areas  and  determine  impact  on  test 
schedule  and  overall  budget. 

Quality  improvements  necessitated  this 
evolution.  The  success  of  future  flight  test 
activity  will  require  the  test  community  share 
the  responsibility  for  risk  management. 
Flight  test  teams  must  incorporate  this 
viewpoint  into  their  program.  The  method 
described  in  this  paper  is  a  proven  system. 
Other  tools  await  development.  Although 
each  application  must  be  tailored  to  the 
program  and  weighed  against  the  costs  and 
the  benefits,  enhanced  control  over  the 
estimate  and  management  of  the  project  will 
result  by  adopting  this  approach.  Future 
flight  test  programs  should  consider 
incorporating  such  a  model  to  assist  in  risk 
management. 
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ABSTRACT 

In  these  times  of  innovation  and  change,  a 
rethinking  of  oversight  and  performance 
measurement  concepts  is  both  important 
and  timely.  An  interagency  team  of 
procurement  professionals  was 
commissioned  by  the  Procurement 
Executive  Association  (PEA)  to  assess  and 
reexamine  the  state  of  the  current  system 
and  to  identify  innovative  approaches  to 
measuring  performance.  The  Procurement 
Measurement  Action  Team  (PMAT) 
devised  a  "new"  methodology  for  assessing 
and  improving  the  health  of  procurement 
systems.  The  Procurement  Performance 
Measurement  Model  focuses  much  more 
attention  on  outcomes  and  results  and 
advocates  the  use  of  benchmarking  and 
reengineering  as  an  effective  process  to 
foster  dramatic  change  in  performance. 

The  model  is,  by  design,  much  different 
from  conventional  assessment  mechanisms 
such  as  procurement  management  reviews. 
It  focuses  on  high  impact  measures.  It  is 
balanced,  and  puts  the  emphasis  on 
prevention  rather  than  detection.  It  is 
customer-oriented  and  cross-functional  in 
that  it  is  not  limited  to  areas  of  direct 
control. 


INTRODUCTION 

On  March  17,  1982,  President  Reagan 
signed  Executive  Order  12352,  "Federal 
Procurement  Reforms"  and  tasked  each 
agency  to  ensure  effective  spending  of 
public  funds.  The  executive  order  required 
Federal  departments  and  agencies  to, 
among  other  things,  name  a  procurement 
executive  who  would  be  required  to 
establish  criteria  and  performance  standards 
for  evaluating  the  respective  agency's 
procurement  system  and  for  certifying  the 
adequacy  of  such  systems.  Subsequently, 
system  criteria  guidelines  were  established 
by  the  Office  of  Federal  Procurement 
Policy  (OFPP)  in  1984  to  serve  as 
guidelines  for  agencies  in  establishing 
criteria  for  their  procurement  systems. 

System  evaluation  criteria  became  the 
enabling  methodology  for  department  and 
agency  system  certifications  through  on¬ 
going  "Acquisition"  or  "Procurement 
Management  Review"  (PMR)  programs. 
Most  agencies  utilize  PMRs  of  procurement 
organizations  to  determine  compliance  with 
these  established  criteria  and  to  certify  the 
adequacy  of  the  procurement  system. 
However,  this  method  is  expensive  and 
time  consuming  and  focuses  on  compliance 
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rather  than  focusing  on  the  desired  results 
of  agency  procurement  systems. 
Additionally,  this  method  has  not  proven 
particularly  effective  in  obtaining  dramatic 
and  sustained  improvement  in  the  quality  of 
the  procurement  operations. 

More  recently,  Vice  President  Gore's 
National  Performance  Review  (NPR) 
established  a  goal  of  moving  from  red  tape 
to  results  to  create  a  Government  that 
works  better  and  costs  less.  NPR  guiding 
principals  envision  four  key  objectives:  (1) 
Cutting  red  tape,  (2)  Putting  customers 
first,  (3)  Empowering  employees  to  get 
results,  and  (4)  Cutting  back  to  basics.  The 
NPR  identified  a  number  of  procurement 
reforms  to  simplify  and  streamline  the 
acquisition  process.  These  reforms  are 
intended  to  radically  decentralize  authority 
to  line  managers,  letting  them  buy  much  of 
what  they  need,  and  simplify  procurement 
regulations  and  processes. 

On  August  3,  1993,  the  President  signed 
the  Government  Performance  and  Results 
Act  of  1993.  It  requires  agencies  to  devise 
five  year  strategic  plans  setting  forth  their 
mission  and  long  term  goals.  Agencies  are 
also  required  to  establish  annual 
performance  plans  and  to  measure  how  well 
they  actually  perform  in  accomplishing 
agency  goals.  On  September  11,  1993,  the 
President  also  signed  Executive  Order 
12862,  "Setting  Customer  Service 
Standards"  requiring  reform  of  the 
executive  branch's  management  practices 
and  operations  to  provide  service  to  the 
public  that  matches  or  exceeds  the  best 
service  available  in  the  private  sector.  The 
executive  order  requires  all  executive 
departments  and  agencies  that  provide 
significant  service  to  the  public  to  survey 
their  customers  and  to  benchmark  customer 


service  performance  against  the  best  in 
business. 

Recognizing  the  need  for  innovation  and 
change,  the  Procurement  Executive 
Association  (PEA)  commissioned  a 
working  group  composed  of  civilian  agency 
procurement  personnel  to  assess  the  state  of 
the  current  system;  identify  innovative 
approaches  to  measuring  performance;  and 
to  develop  recommendations  for  measuring 
the  health  of  agency  acquisition  systems. 
The  PMAT  consists  of  representatives  from 
Treasury,  Transportation,  General  Services 
Administration,  Commerce,  Health  and 
Human  Services,  and  the  U.S.  Mint.  This 
paper  presents  the  results  of  the  PMAT 
efforts  to  re-invent  the  Federal  performance 
measurement  model. 

PERFORMANCE  MEASUREMENT 
ACTIVITY 

The  PMAT  began  by  conducting  basic 
research  in  performance  measurement. 
Research  conducted  by  the  PMAT  consisted 
of  general  readings,  site  visits  to  industry, 
contacts  with  state  and  local  governments, 
academia  and  formal  classroom  training  in 
developing  performance  measures  and  in 
benchmarking.  Contact  with  industry 
revealed  the  use  of  operational  performance 
measures  in  combination  with  the  more 
common  and  traditional  financial  measures. 
Customer  input  was  also  sought,  measured 
and  used  as  a  tool  for  improving 
performance.  Differences  in  the 
fundamental  mission  of  the  purchasing 
activities  in  government  and  the  private 
sector  were  found  to  be  negligible. 
Industry's  growing  use  of  benchmarking  as 
a  measurement  tool  was  also  noted  and 
documented  including  the  requirement  for 
an  established  benchmarking  program  as  a 
prerequisite  for  the  Malcolm  Baldrige 
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Award.  Interest  in  benchmarking  is  likely 
to  grow  exponentially,  eventually  becoming 
standard  business  practice. 

Benchmarking  is  the  continuous  process  of 
analyzing,  measuring  and  assessing  an 
organization's  performance  against  the 
"best"  for  the  purpose  of  instituting 
dramatic  and  sustained  performance.  The 
International  Benchmarking  Clearinghouse, 
a  service  of  the  American  Productivity  and 
Quality  Center  in  Houston,  Texas,  which 
has  researched  and  written  extensively  on 
the  use  of  benchmarking,  defines 
benchmarking  "as  an  ongoing  measurement 
and  analysis  process  that  compares  internal 
practices,  processes  or  methodologies  with 
those  of  other  organizations."  The  purpose 
of  benchmarking  studies  is  to  identify  best 
practices  that  may  be  adapted  to  a  wide 
range  of  organizations.  The  identification 
and  use  of  best  practices  leads  to  the 
dramatic  process  improvements  associated 
with  benchmarking  efforts.  (1) 

PMAT  examined  performance 
measurement  practices  at  Eastman  Kodak 
and  Texas  Instruments.  "Procurement 
benchmarking"  conducted  by  the  Center  for 
Advanced  Purchasing  Studies  (CAPS) 
associated  with  the  College  of  Business  at 
Arizona  State  University  was  also  studied. 
Eastman  Kodak  Company  is  benchmarking 
purchasing  functions  world-wide.  A  self- 
assessment  survey  is  distributed  to  and 
completed  by  Kodak's  purchasing  units  on 
an  annual  basis.  At  this  time,  Kodak 
measures  performance  against  established 
critical  success  factors.  They  employ  both 
qualitative  and  quantitative  measures.  The 
results  of  the  effort  are  used  in  developing 
improvement  plans  through  the  use  of  best 
practices. 


Texas  Instruments  is  benchmarking  both 
internally  and  externally.  Purchasing  serves 
as  a  component  on  a  team  approach.  Texas 
Instruments  relies  upon  varied  forms  of 
information  collection  instruments.  CAPS 
was  established  to  respond  to  the  needs  of 
private  sector  purchasing.  They  have  also 
expanded  their  benchmarking  services  to 
state  and  local  governments.  CAPS  is 
supported  by  corporate  grants.  They  are 
benchmarking  purchasing  in  26  industries, 
including  state  and  local  governments. 
Industry  identifies  specific  benchmarks  to 
be  used  in  addition  to  standard  generic 
measures  (e.g.  timeliness,  cost 
effectiveness,  etc.).  Many  of  the  metrics 
have  applicability  to  Federal  purchasing 
and  may  prove  beneficial  in  comparing 
Federal  procurement  with  private  sector 
and  state  and  local  government 
procurement. 

In  developing  the  Procurement 
Performance  Measurement  Model,  the 
PMAT  was  heavily  influenced  by  an  article 
entitled  "The  Balanced  Scorecard-Measures 
that  Drive  Performance"  by  Robert  C. 
Kaplan  and  David  P.  Norton  in  the 
January/February  1992  edition  of  the 
Harvard  Business  Review.  Kaplan  and 
Norton  pointed  out  the  need  for  a  balanced 
presentation  of  both  financial  and 
operational  measures.  Kaplan  and  Norton 
devised  a  "balanced  scorecard"  -  a  set  of 
measures  from  four  different  perspectives 
that  gives  managers  a  fast  but 
comprehensive  view  of  the  business.  The 
authors  likened  the  "balanced  scorecard"  to 
dials  and  indicators  relied  upon  in 
navigating  an  airplane.  A  pilot  needs  to 
rely  on  many  indicators  simultaneously  to 
ensure  the  steady  flight  of  the  airplane. 

The  "Balanced  Scorecard"  links 
performance  measures  from  four 
perspectives:  Customer,  Financial,  Internal 
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Business,  and  Innovation  and  Learning. 
Figure  1  depicts  the  Kaplan  and  Norton 
model.  (2) 

PROCUREMENT  PERFORMANCE 
MEASUREMENT  MODEL 

The  Procurement  Performance 
Measurement  Model  developed  by  the 
PMAT  is  presented  in  Figure  2.  The 
model  was  designed  to  assess  the  primary 
role  of  the  procurement  system:  to  provide 
a  quality  product  or  service  at  a  good  price 
within  timeframes  necessitated  by  the 
customer.  The  model  uses  the  Kaplan  and 
Norton  balanced  scorecard,  with  some 
modification,  to  link  performance  measures 
from  five  perspectives:  Financial, 
Customer,  Internal  Business,  Innovation 
and  Learning  and  Employee 
Empowerment.  Within  the  five 
perspectives,  eleven  performance  goals  are 
identified  as  critical  success  factors  for 
procurement  organizations.  Key  measures 
and  data  collection  methods  are  identified 
for  each  goal.  Each  goal  has  at  least  one 
corresponding  counter  balance  goal.  For 
example,  successful  accomplishment  of 
"Timeliness"  may  impact  upon 
organizational  performance  in  a  number  of 
other  goals,  including  but  not  limited  to: 
"Quality  of  product",  "Acquisition 
Excellence",  or  "Quality  Work 
Environment" . 

In  developing  the  model,  each  goal  and 
corresponding  measure  is  well  defined  by 
PMAT.  In  many  cases  the  goal  and 
measure  are  self-explanatory.  "Acquisition 
Excellence"  speaks  to  the  optimum  quality 
of  the  procurement  process.  The  measure 
assesses  the  extent  to  which  internal  quality 
systems  (designed  to  ensure  an  effective 
and  efficient  acquisition  process)  are  in 
place.  "Maximize  Value  at  the  Least  Cost" 


is  intended  to  maximize  the  value  of  the 
procurement  system  relative  to  purchasing 
costs.  The  cost  to  spend  ratio  (i.e., 
purchasing  costs  as  a  percentage  of  contract 
dollars  spent)  is  used  as  the  key  measure  in 
assessing  the  cost-effectiveness  of  the 
procurement  process.  "Maximize  Cost 
Savings"  relates  to  the  verifiable  savings  or 
avoidance  of  cost  through  the  deliberate 
efforts  of  the  procurement  office.  The  ratio 
of  dollars  saved  or  avoided  as  a  percentage 
of  contract  dollars  obligated  is  used  as  the 
key  measure. 

The  model  was  designed  to  be  relatively 
simple  and  inexpensive  to  conduct  a 
comparative  assessment  of  organizational 
performance.  Four  methods  of  data 
collection  are  used.  A  Customer  Survey  is 
used  to  collect  internal  customer 
perceptions  relating  to  timeliness  of  product 
(e.g.  on-time  delivery  defined  by  the 
customer),  quality  of  product  (e.g.  quality 
of  product  as  defined  by  the  customer),  and 
responsiveness  of  procurement  to  customer 
needs.  An  Employee  Survey  is  given  to  all 
employees  of  the  procurement  organization 
to  assess  perceptions  relating  to  the  work 
environment  and  executive  leadership.  A 
Self  Assessment  is  completed  by  the  senior 
procurement  official  in  the  organization  to 
rate  and  assess  performance  against  Quality 
Workforce,  Mission  Planning,  Acquisition 
Excellence,  and  Data  Collection  goals. 
Statistics  obtained  from  the  Federal 
Procurement  Data  System  and  the 
procurement  organization  are  used  to 
develop  the  cost  to  spend  ratio  and 
purchasing  influenced  savings. 

The  PMAT  used  the  services  of  experts, 
such  as  the  Joint  University  of  Maryland- 
University  of  Michigan  program  in  Survey 
Methodology  to  develop  the  Customer  and 
Employee  Surveys.  Both  surveys  use  a 
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standard  approach  to  data  collection. 
Respondents  are  asked  to  rate  their  level  of 
agreement  and  level  of  importance 
associated  with  each  survey  statement.  Self 
Assessments  developed  by  Eastman  Kodak 
Company  and  Texas  Instruments  were  used 
as  a  model  for  PM  AT  self  assessments. 

The  Self  Assessment  survey  contains  a 
series  of  specific  measurable  statements 
associated  with  a  performance  goal  (e.g. 
Quality  Workforce).  The  statements  reflect 
differing  degrees  of  performance  with  a 
particular  subject  matter.  The  senior 
procurement  official  selects  the  statement(s) 
reflective  of  his  organizations  performance. 
A  rating  (from  1  to  5)  is  subsequently 
assigned  to  each  response. 

A  methodology  for  using  the  model  as  an 
effective  tool  to  measure  and  enable 
continuous  change  was  developed  by 
PM  AT.  Information  collection  and 
presentation  of  results  is  primary  research. 
Once  complete,  the  information  is  used  to 
ascertain  how  an  organization  compares 
with  other  organizations  and  identification 
of  best-in-class,  in  eleven  performance 
areas.  The  organization  is  then  free  to 
identify  opportunities  for  improvement. 
Benchmarking  identifies  best  practices. 
Benchmarking  with  the  best-in-class 
accelerates  the  process  of  change  and 
fosters  breakthrough  improvements. 

After  the  initial  development  of  the 
Procurement  Performance  Measurement 
Model,  the  PM  AT  conducted  a  BETA  test 
in  fourteen  procurement  offices  in  six 
civilian  agencies.  The  purpose  of  the  test 
was  to  validate  the  performance  goals, 
measures  and  the  survey  instruments.  Over 
500  employees,  in  their  capacities  as 
internal  customer  or  procurement 
personnel,  participated  in  the  test.  To 
ensure  consistency  in  the  deployment  of  the 


test,  PMAT  teams  followed  a  very 
structured  approach  in  presenting  and 
administrating  the  data  collection.  An 
executive  briefing  was  provided  to  senior 
management  on  the  purpose  and  scope  of 
the  test.  PMAT  provided  customer  and 
procurement  staff  with  a  briefing  on  the 
Procurement  Performance  Measurement 
Model,  how  it  was  developed  and  its 
intended  purpose.  An  overview  on  the 
process  of  benchmarking  was  also 
provided.  At  the  conclusion  of  the 
briefings,  the  employee  and  customer 
surveys  were  administered.  The  senior 
procurement  official  was  tasked  with 
completing  the  self-assessment. 

Unlike  a  typical  PMR,  PMAT  performed 
data  analysis  on-site  and  provided  all 
participants  with  test  results  within  a  day  of 
data  collection.  A  glyph  was  used  to 
present  a  report  card  for  each  procuring 
organization  tested.  A  glyph  is  a  tool  used 
in  Total  Quality  Management  to  evaluate 
several  different  alternatives  based  on  three 
or  more  quality  characteristics.  PMAT's 
approach  to  presenting  the  results  in  a 
glyph  was  reflective  of  performance 
measurement  work  conducted  by  Eastman 
Kodak  Company.  Figure  3  is  a 
representation  of  a  PMAT  glyph.  The 
implications  of  the  report  card  will  be 
discussed  shortly.  PMAT  presented  the 
results  of  the  data  collection  in  focus 
groups  held  jointly  with  customers  and 
procurement  staff.  The  focus  groups  also 
provided  excellent  feedback  to  the  PMAT 
on  the  utility  of  the  Performance 
Measurement  Model,  including  data 
collection  instruments. 

As  a  result  of  the  testing  and  obtaining 
feedback  in  focus  groups  and  other  sources, 
PMAT  concluded  that  the  goals  reflected 
the  critical  success  factors  of  the 
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procurement  organizations.  Participants 
observed  that  the  model  measured  the  right 
things.  PMAT  also  found  the  performance 
measures  valid  in  assessing  achievement  of 
performance  goals.  The  majority  of  test 
participants  indicated  that  the  assessment 
reflected  their  perceptions  of  their 
organizations  performance.  After 
conducting  the  test,  PMAT  concluded  that 
the  Procurement  Performance  Measurement 
Model,  with  minor  adjustments  identified 
during  the  test,  could  provide  an  effective 
tool  that  measures  the  overall  health  of  the 
procurement  organization.  The  model 
provides  specific  measures  which  indicate 
superior  performance  as  well  as  areas  that 
could  be  targeted  for  improvement  based 
upon  input  from  management,  employees 
and  customers. 

DATA  ANALYSIS  AND  IMPROVING 
PERFORMANCE 

The  data  obtained  from  the  customer 
survey,  employee  survey,  self  assessment 
and  statistics  are  all  considered  to 
determine  how  effectively  a  procurement 
organization  met  the  performance  goals. 
Upon  conclusion  of  the  test,  PMAT 
collected  and  normalized  the  data  from  all 
test  sites.  Figure  3  presents  the  results  of 
the  data  collection  for  one  of  the  procuring 
organizations.  The  data  is  normalized. 

The  glyph  is  reporting  on  how  the 
organization  stacks  up  with  other 
procurement  organizations  in  thirteen 
performance  areas.  The  closer  to  the 
center  of  the  glyph  the  better  the 
performance.  The  shaded  area  represents 
the  performance  outcome  for  organization 
"A".  The  dotted  line  represents  the 
average  of  all  tests  sites. 

In  the  area  of  customer  perspective, 
organization  "A"  is  doing  quite  well. 


Organization  "A's"  internal  customers  are 
most  satisfied  with  the  timeliness  of 
product  or  services;  quality  of  the  product 
or  services  and  with  the  responsiveness  of 
the  procurement  organization.  In  fact,  this 
organization  may  be  best  in  class  for 
Customer  perspective.  Employee 
perspective  is  reporting  mixed  results. 

Under  "Executive  Leadership"  employee 
perceptions  indicate  operations  are 
conducted  in  an  efficient,  professional  and 
ethical  manner.  Nonetheless,  there  may  be 
opportunities  for  improvement  with  the 
quality  of  the  work  environment.  There 
may  also  be  targets  for  improvement  under 
"Acquisition  Excellence",  "Accurate  and 
Timely  Data  Collection",  and  the  goals 
under  the  Financial  Perspective. 

As  previously  discussed,  conducting  the 
assessment  is  designed  to  be  the  first  step  in 
improving  performance  by  providing 
information  on  how  the  organization 
compares  with  other  procurement 
organizations  for  the  purpose  of  identifying 
best  practices  in  critical  success  areas.  The 
assessment  is  primary  information 
collection.  Procurement  organizations  use 
this  information  to  immediately  identify 
areas  of  weakness  and  begin  efforts  to 
review  their  processes.  This  approach  to 
improvement  is  participatory  and  empowers 
the  organization  to  identify  what's 
important.  Process  mapping  will  assist  the 
organization  to  better  understand  the 
process,  identify  areas  of  high  impact  that 
need  improvement  or  reengineering, 
streamline  the  process  by  immediate 
elimination  of  duplication  or  obvious 
inefficiencies  and  identify  potential 
benchmarking  topics.  Comparisons  to  the 
best-in-class  will  identify  what  is  possible. 
Benchmarking  with  the  best  in  class  will 
permit  breakthrough  improvements  and 
establishing  goals  of  becoming  best-in- 
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class.  The  benefits  of  benchmarking  are 
enormous.  It  allows  the  organization  to 
accelerate  the  process  of  change  and 
accomplish  breakthrough  improvements.  It 
adopts  best  practices  of  organizations 
recognized  for  excellence  and  improves 
upon  them. 

As  noted  above,  benchmarking  is  viewed  as 
a  key  critical  component  of  the  PMAT 
Procurement  Performance  Measurement 
Model.  PMAT  also  developed  a  "tool  box" 
for  use  by  the  practitioner  in  effecting 
change.  Each  performance  goal  is 
supported  by  a  key  measure  and  tools  for 
further  analysis.  Figure  4  represents  the 
"Timeliness"  goal  and  tools  for  further 
analysis.  In  cases  where  the  "timeliness" 
key  measure  indicates  improvement  is 
needed,  the  organization  should  carefully 
examine  the  process  for  possible 
benchmarking  or  rengineering  efforts. 

Cycle  time  or  procurement  administrative 
lead-time  may  also  be  used  for  primary 
data  analysis. 

IMPLEMENTATION 

The  PMAT  Procurement  Performance 
Measurement  Model  is  being  implemented 
in  a  number  of  Federal  departments  and 
agencies  as  a  tool  in  assessing  and 
improving  the  health  of  procurement 
organizations.  To  be  successful  in 
establishing  a  sustained  program,  PMAT 
recommends  the  following  activities: 

Coordinated  Performance  Data  Collection. 
This  entails  the  distribution  of  data 
collection  instruments  to  participating 
organizations.  A  program  for  just  in  time 
training  should  be  established  for  all 
participants.  Briefings  on  the  purpose  and 
scope  of  the  model  and  benchmarking 


should  be  provided  prior  to  administration 
of  the  survey  instruments. 

Data  Analysis.  A  central  activity  should 
receive  and  analyze  data  reports  from 
individual  procuring  organizations.  Data 
should  be  normalized  and  results  presented 
in  the  PMAT  glyph  reporting  the 
organization's  performance  against  the 
measure;  mean  performance  of  all 
organizations  participating  in  the  study;  and 
best-in-class  performance. 

Benchmarking.  Participating  organizations 
should  be  encouraged  to  participate  in 
benchmarking  symposiums  and  to  use 
benchmarking  to  improve  performance. 

Recalibrate.  Recalibrate,  refine  and  adjust 
performance  model.  Identify  new 
benchmarking  opportunities. 

CONCLUSIONS 

In  late  1994,  the  PMAT  reported  to  the 
PEA  that  the  final  model  can  be  an 
effective  tool  in  implementing  a  results 
oriented  approach  in  assessing  agency 
procurement  systems.  This  model  is,  by 
design,  much  different  from  conventional 
assessment  mechanisms  such  as  PMRs. 

The  Procurement  Performance 
Measurement  Model: 

•  Focuses  on  high  impact  measures. 

•  Is  intended  to  be  easy  and 
economical  to  use. 

•  Is  balanced  and  puts  the  emphasis 
on  prevention  rather  than  detection. 

•  Is  customer  oriented. 
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•  Empowers  the  organization  to 
identify  opportunities  for 
improvement. 

•  Provides  performance  data  on 
procuring  organizations  annually. 

•  Provides  for  employee  input  or 
perspective. 

•  Recognizes  and  rewards  superior 
performance. 

PMAT  also  believes  that  the  model  can  be 
effective  in  responding  to  results  oriented 
performance  requirements  contained  in  the 
Federal  Acquisition  and  Streamlining  Act 
of  1994  (FAS A)  and  Executive  Order 
12931  as  well  as  the  customer  focus  and 
best  practices  contemplated  by  Executive 
Order  12862.  The  FASA  requires  that 
OFPP  develop  results-oriented  acquisition 
process  guidelines.  Executive  Order 
12931,  "Procurement  Reform",  replaces 
Executive  Order  12352  and  contains  a 
number  of  requirements  that  the  model 
would  satisfy.  Such  concerns  as  focusing 
more  attention  on  meeting  customer  needs, 
developing  procurement  goals,  measuring 
procurement  offices  against  those  goals, 
promoting  innovation  and  rewarding 
performance  are  all  addressed  by  the 
model. 

END  NOTES 

1.  Productivity  Press,  Inc.,  The 
Benchmarking  Management  Guide, 
(1993),  pp  4  and  5. 

2.  Harvard  Business  Review,  The 
Balanced  Scorecard  -  Measures  that 
Drive  Performance,  January-February 
1992,  pp  71  -  79. 
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The  Balanced  Scorecard  Links  Performance  Measures 


Figure  2 
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Pursuit  of  Excellence  In  Acquisition 
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Figure  3 
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CUSTOMER  PERSPECTIVE 
TIMELINESS 


GOAL:  To  provide  goods  and  services  that  meet  user  needs  in  a  timely  manner. 


The  mission  or  objective  of  the  acquisition  process  is  to  obtain  quality  goods  and  services 
that  meet  user  needs  in  a  timely  manner  and  at  a  reasonable  cost.  Timeliness  of  the 
acquisition  process  is  a  major  concern  of  management  as  it  directly  affects  the  agency's 
ability  to  meet  it's  mission  goals.  To  succeed  in  conducting  timely  procurements,  it  is 
essential  to  plan  for  the  acquisition  of  goods  and  services  needed  by  the  government.  The 
need  for  procurement  planning  should  be  considered  for  every  proposed  acquisition  and  a 
plan  developed  where  appropriate.  Planning  activities  should  encompass  the  entire 
acquisition  process  from  the  inception  of  a  program  to  completion  of  the  contract. 
Coordination  between  program  and  procurement  personnel  on  specific  requirements,  lead 
times  and  milestones  is  essential  to  ensure  that  both  technical  and  business  aspects  are 
considered. 

KEY  MEASURE:  Customer  satisfaction  should  be  used  as  the  key  measure  in  assessing 
the  performance  of  the  procurement  system  in  providing  goods  and  services  when  they  are 
needed.  A  customer  survey  can  be  used  to  obtain  the  perceptions  of  the  customer.  For 
purposes  of  the  survey,  customer  may  include  internal  users,  external  users  or  both.  The 
survey  instrument  should  focus  on: 

=>  Customer's  degree  of  satisfaction  with  the  procurement  process  in  delivering  products 
or  services  on  time  to  meet  the  needs  of  the  agency.  On  time  delivery  is  defined  by  the 
customer. 

=>  Customer's  degree  of  satisfaction  in  planning  activities  including  the  establishment  of 
lead  times  and  acquisition  milestones. 

=>  Customer's  degree  of  satisfaction  with  on-going  communication  with  procurement  staff 
throughout  the  procurement  cycle  on  lead  time  and  milestone  status. 

The  key  measure  results  will  be  used  to  display  the  organization’s  performance  relative  to 
the  best-in-clkass  and  average  baseline  performance. 

TOOLS  FOR  FURTHER  ANALYSIS:  In  cases  where  key  measures  indicate  improvement 
is  needed,  the  organization  should  carefully  esaminte  the  process  for  possible 
benchmarking  or  reengineering  efforts. 

Processing  time  can  be  measured  especially  when  customer  satisfaction  rates  are  low.  It 
can  be  measured  and  assessed  either  from  the  standpoint  of  cycle  time  or  procurement 
administrative  leadtime.  Cycle  time  is  the  time  from  program  inception  to  delivery  of 
product  or  service.  Cycle  time  includes  consideration  of  mission  needs,  planning, 
alternatives,  budgetary  programming,  needs  descriptions  and  acquisition.  Procurement 
administrative  leadtime  represents  the  time  from  receipt  of  a  complete  and  proper 
requisition  through  contract  award.  In  either  cases,  processing  time  can  also  be  compared 
by  benchmarking  accomplishments  over  time  or  by  comparisons  with  other  similar 
operations. 


Figure  4 
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SLAD:  A  SUCCESS  STORY  IN  REENGINEEERING  THE  ACQUISITION  PROCESS 


Michael  W.  Starks 
U.S.  Army  Research  Laboratory 


ABSTRACT 

The  situation  with  respect  to 
Survivability  /Lethality  Analysis  in  the  U.S. 
Army  prior  to  1992  is  described.  Significant 
problems  with  that  situation  are  enumerated. 
The  formation  of  SLAD  is  exhibited  not  only 
as  a  solution  to  those  problems,  but  as  a  more 
general  examplar  of  a  success  in  reengineering 
the  acquisition  process. 

INTRODUCTION 

I  will  show  that  the  Survivability/Lethality 
Analysis  Directorate  (SLAD)  of  the  Army 
Research  Laboratory  is  a  success  story  with 
respect  to  Reengineering  the  Acquisition  Pro¬ 
cess.  The  historical  situation  with  respect  to 
the  Army’s  process  for  conducting 
survivability /lethality  analyses  will  be 
described  and  problems  with  that  process  will 
be  noted.  SLAD’s  organization  and  operating 
process  will  be  described  as  a  response  to  the 
historical  problems.  Most  of  the  paper  will 
cover  some  of  the  specific  lessons  —  technical 
and  organizational  —  that  the  Army  has 
learned  through  transition  to  a  re-engineered 
process. 

Before  the  formation  of  SLAD,  it  was 
extremely  awkward  and  excessively  costly  for 
Army  Project  Managers  (PMs)  to  obtain  the 
help  they  needed  to  ensure  that  the  materiel 
they  were  responsible  for  developing  was 
properly  optimized  to  ensure  survivability  on 
the  modern  battlefield.  For  survivability  in 
the  face  of  electronic  warfare  (EW)  threats, 
the  premiere  organization  was  the  Vulnerabil¬ 
ity  Assessment  Laboratory  (VAL)  of  White 
Sands  Missile  Range  (WSMR).  For  nuclear 
threats,  the  responsible  entity  was  the  Harry 
Diamond  Laboratory  (HDL)  in  Adelphi, 
Maryland.  For  ballistic  threats,  the  provider 
was  the  Ballistic  Research  Laboratory  (BRL) 
of  Aberdeen  Proving  Ground.  For  chemical 
and  biological  threats,  including  susceptibility 
to  smoke,  the  Chemical  Research  and 


Development  Engineering  Center  (CRDEC)  at 
Edgewood  Arsenal  was  the  place  to  go.  Each 
of  the  organizational  entities  mentioned  thus 
far  was  a  hands-on  provider  of  survivability 
analysis  to  Army  PMs  and  to  the  Acquisition 
Process.  In  addition,  there  was  a  Survivabil¬ 
ity  Management  Office  (SMO)  and  a  Vulnera¬ 
bility  Lethality  Assessment  Management 
Office  (VLAMO);  these  offices  were  trying  to 
perform  the  Herculean  task  of  coordinating 
the  various  survivability  needs. 

Unmentioned  so  far  are  the  providers  of  sur¬ 
vivability  technology  in  the  Army’s  various 
Research  Development  and  Engineering 
Centers  (RDECs)  as  well  as  Test  and  Evalua¬ 
tion  Command  (TECOM)  which  is  responsi¬ 
ble  for  some  of  the  Army’s  survivability  test¬ 
ing.  In  addition,  Training  and  Doctrine  Com¬ 
mand  (TRADOC)  centers  and  schools  are 
responsible  for  developing  survivability 
requirements  for  Army  systems.  The  surviva¬ 
bility  situation  was  sufficiently  confusing  that 
in  1991  the  Deputy  Commanding  General  of 
the  Army  Materiel  Command  commissioned  a 
major  outside  review  of  what  was  then  called 
the  Vulnerability/  Lethality/Survivability 
(VLS)  methodology  and  organizational  struc¬ 
ture.  One  of  the  graphics  from  that  review  is 
shown  in  Figure  1.  It  is  clearly  not  a  pretty 
picture. 

Exacerbating  the  organizational  diffusion  was 
funding  inconsistency.  Virtually  every  kind 
of  Program  6  funds  were  used  in  support  of 
the  activities  I  have  mentioned.  Moreover, 
some  of  the  organizations  mentioned  had  sub¬ 
stantial  mission  funding  for  conduct  of  their 
work  while  other  organizations  had  almost 
none. 

A  final  problem  with  the  pre-SLAD  environ¬ 
ment  was  more  subtle,  but  also  critically 
important.  Some  of  the  pre-SLAD  organiza¬ 
tions  had  the  attitude  that  they  were  part  of 
the  PM’s  team  and  that  their  professional 
goal  was  to  help  the  PM  field  the  most  sur- 
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vivable  materiel  possible.  Other  organiza¬ 
tions  had  the  attitude  that  they  were  part  of 
the  Department  of  the  Army  team  which 
judged  whether  or  not  the  PM  had  taken  all 
appropriate  action  to  ensure  system  surviva¬ 
bility.  This  clash  of  cultures  was  not  only 
frustrating  to  PMs,  it  also  caused  a  fair 
amount  of  sniping  across  organizational 
boundaries,  and  a  lack  of  uniformity  with 
respect  to  supporting  DA  goals. 

SLAD  ORGANIZATION 

When  SLAD  was  created,  we  aimed  to  pro¬ 
vide  one-stop  shopping  for  all  SLA  needs. 
Although  the  specific  expertise  in  the  various 
survivability  disciplines  might  have  to  remain 
geographically  dispersed,  our  goal  was  to 
make  this  geographical  and  discipline  diver¬ 
sity  transparent  to  our  customers.  As  subsi¬ 
diary  goals  we  also  aimed  to  rationalize  the 
funding  for  SLAD  and  to  build  a  single  cul¬ 
ture  (Help  PMs?  Judge  PMs?)  for  conduct  of 
SLA. 

Our  primary  vehicle  for  achieving  all  three  of 
these  goals  was  integrated  analyses ,  the  plan¬ 
ning  and  resourcing  of  which  was  to  be  imple¬ 
mented  by  Integrated  Analysis  Teams.  The 
organizational  structure  we  created  to  sup¬ 
port  integrated  analysis  is  shown  in  Figure  2. 
The  ovals  indicate  the  pre-SLAD  organiza¬ 
tions  who  contributed  the  500  civil  servants, 
50  military,  and  200  +  contractor  support 
personnel  who  staffed  SLAD  at  its  inception 
in  October  1992.  Note  that  the  organization 
is  aligned  by  technical  discipline.  For  pur¬ 
poses  of  planning  and  resourcing,  however,  it 
is  helpful  to  picture  the  organization  as  it  is 
shown  in  Figure  3.  Each  area  staffer  in  the 
Integration  Office  leads  an  inter-disciplinary 
integrated  analysis  team  which  typically  is 
populated  by  cost  center  managers  —  Branch 
Chiefs  —  from  the  three  line  Divisions.  The 
teams  take  the  dollar  guidance  for  mission 
funding  they  have  been  given  by  senior 
management  and  conduct  a  complex  planning 
process  for  the  systems  in  their  purview. 
Requirements  of  the  supported  PMs  and  of 
the  Independent  Evaluators  are  sought,  as  are 
the  emphasis  areas  from  HQDA.  The  teams 
allocate  money  versus  systems,  and  also  must 
further  allocate  the  money  against  discipline 
—  specific  tasks  in  support  of  those  systems. 


Programs  thus  developed  are  approved  or 
modified  by  an  Executive  Steering  Committee 
consisting  of  the  three  SLAD  Division  Chiefs 
and  the  Chief  of  the  Integration  Office. 

During  the  planning  that  led  to  development 
of  this  process,  we  were  also  able  to  bridge 
the  cultural  gap  between  the  organizations 
which  regarded  themselves  as  helpers  of  PMs 
and  those  with  a  self-image  as  evaluators  of 
PMs.  We  did  this  by  chartering  a  Process 
Action  Team  (PAT)  with  representatives 
from  PEOs,  from  the  Independent  Technical 
Evaluator,  and  from  SLAD.  The  PAT 
developed  a  very  detailed  process  (Figure  4), 
according  to  which  SLAD  simultaneously 
makes  its  data  available  to  the  PM  (as  helper) 
and  to  the  evaluators.  Explicit  dispute  reso¬ 
lution  procedures  were  also  included  in  the 
process. 

It  is  worth  lingering  a  moment  on  the  plan¬ 
ning  scheme  I  have  outlined  to  make  it  com¬ 
pletely  clear  how  it  tends  to  support  achieve¬ 
ment  of  the  goals  earlier  stated.  First,  the 
integrated  planning  process  clearly  provides 
the  sought  after  one-stop  shopping  for  Army 
PMs.  All  a  PM  needs  to  know  in  order  to 
access  needed  SLAD  support  is  the  name  of 
the  Integration  Office  person  responsible  for 
his  mission  area.  Second,  the  process  pro¬ 
vides  the  sought  after  funding  rationalization. 
On  a  global  basis,  SLAD  funding  guidance  is 
designed  both  to  provide  more  uniform  mis¬ 
sion  funding  support  across  Integration  Areas 
and  to  obtain  more  uniform  customer  support 
as  well. 

It  is  also  worth  emphasizing  that  the 
integrated  planning  process  just  described, 
and  the  organizational  structure  which  sup¬ 
ports  it,  was  very  carefully  designed  by 
experienced  and  senior  level  practitioners  of 
the  various  survivability  disciplines.  This 
point  is  important  in  view  of  the  successes  I 
will  attribute  to  the  SLAD  process  and  struc¬ 
ture  below.  Some  organizational  structures 
make  abstract  sense  when  viewed  from  a 
sufficiently  high-level  perspective,  but  make 
considerably  less  sense  when  exposed  to  the 
crucible  of  practical  application.  It  is  critical 
to  involve  experienced  subject  matter  experts 
in  organizational  re-engineering  both  to 
ensure  relevance  of  organizational  structure 
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NOTE:  ISSUES/RESULTS  WILL  BE  COORDINATED  WITH  THE  TIWG  WHERE  APPROPRIATE 


and  to  guarantee  that  new  organization  struc¬ 
tures  will  be  properly  responsive  to  real-world 
business  problems. 

TECHNICAL  BENEFIT  OF  SLAD 

We  expected  to  achieve  significant  new  techn¬ 
ical  synergies  when  SLAD  was  formed  and  we 
did;  I’ll  give  an  example  below.  However,  we 
also  obtained  substantial  synergies  which  we 
had  not  predicted  in  detail;  some  of  these 
have  greater  potential  long-term  significance 
than  the  ones  we  did  expect.  I’ll  give  exam¬ 
ples  of  these  too.  The  word  "synergy"  has 
been  used  to  characterize  everything  from 
corporate  mergers  to  snake  oil,  so  it  may  be 
helpful  if  I  indicate  what  we  mean  by  it  in 
SLAD.  .We  use  the  word  to  signal  certain 
kinds  of  operating  efficiencies  that  resulted 
from  the  formation  of  SLAD;  we  also  use  it  to 
describe  certain  kinds  of  product  improve¬ 
ments. 

An  example  of  a  synergy  we  did  expect  when 
we  formed  SLAD  concerns  handoff  of  a  cer¬ 
tain  kind  of  data  from  our  Electronic  War¬ 
fare  Division  (EWD)  to  our  Ballistic  Vulnera¬ 
bility  Lethality  Division  (BVLD).  Histori¬ 
cally,  the  BVLD  assumed  various  simple  (i.e. 
uniform  or  Gaussian)  distributions  of  hits  on 
targets.  Within  SLAD,  however,  EWD  has 
the  capability  of  measuring  or  simulating 
actual  hit  distributions,  both  for  nominal  con¬ 
ditions  and  for  various  types  of  electronic 
countermeasures  (ECM).  Here  is  a  case  where 
EWD’s  output  becomes  BVLD’s  input,  and 
with  a  number  of  desirable  consequences. 
There  are  no  additional  costs,  the  PM  gets  an 
integrated  answer  over  the  electronic  warfare 
and  conventional  ballistic  disciplines,  and  the 
ballistic  output  is  based  on  more  realistic 
input  than  before.  This  last  point  is  particu¬ 
larly  critical  since  the  ballistic  outputs 
("Pks")  resonate  through  the  entire  down¬ 
stream  analysis  process,  and  are  therefore 
critical  to  the  whole  Research,  Development, 
and  Acquisition  (RDA)  process. 

Important  as  realization  of  this  expected  syn¬ 
ergy  is,  it  is  less  important  than  other  techni¬ 
cal  synergies  we  have  gained  which  were 
wholly  unpredicted.  Recall  that  prior  to  the 
formation  of  SLAD,  there  were  distinct 
organizational  entities  responsible  for  the 


various  technical  disciplines:  conventional 
ballistics,  electronic  warfare,  chemical, 
nuclear,  biological,  smoke,  and  environmen¬ 
tal.  Each  of  these  organizations  had  a  discer- 
nibly  different  culture.  There  were  different 
ways  of  posing  the  global  survivability  prob¬ 
lem,  different  tools,  techniques,  and  metho¬ 
dologies  for  solving  survivability  problems, 
different  vocabularies  for  discussing  surviva¬ 
bility  problems,  and  different  views  as  to  who 
the  customer  set  was  for  survivability  pro¬ 
ducts. 

Experienced  leaders  will  recognize  that  it  is 
not  possible  to  overcome  cultural  differences 
of  the  kind  enumerated  by  senior  manage¬ 
ment  edict.  What  has  happened  is  that  the 
process  we  developed  for  planning  the  SLAD 
program  has  inherently  changed  the  culture 
in  hundreds  of  ways,  some  small  and  some 
large.  Recall  that  the  SLAD  planning  process 
features  multi-disciplinary  teams  led  by  a 
Senior  Project  Officer  from  the  Integration 
Office,  with  Branch  Chiefs  representing  each 
SLAD  discipline.  During  SLAD’s  short  life¬ 
time  of  just  over  2  years,  that  process  has  led 
to  remarkable  progress  towards  a  unified 
SLAD  culture  and  to  technical  synergies  that 
could  not  have  been  predicted  when  SLAD 
was  created. 

The  fundamental  job  of  the  planning  teams  is 
to  develop  and  prioritize  the  work  packages 
supporting  each  Army  system.  This  requires, 
for  example,  that  the  teams  evaluate  the  rela¬ 
tive  importance  of  an  electronic  warfare  work 
package  involving  CRUSADER’s  signature,  a 
conventional  ballistics  work  package  involv¬ 
ing  alternative  armor  packages,  an  NBC  work 
package  concerning  collective  protection  sys¬ 
tems,  and  an  environmental  work  package 
concerning  the  capability  of  CRUSADER’s 
GPS  receiver  as  a  function  of  precipitation 
rate.  Clearly,  a  necessary  condition  on  suc¬ 
cessful  accomplishment  of  the  prioritization 
task  is  that  the  team  members  understand 
what  is  to  be  done  in  each  workpackage  and 
why.  In  turn,  this  requires  that  each  team 
member  understand  —  not  necessarily 
embrace  —  a  fair  amount  of  the  technical  cul¬ 
ture  of  each  SLAD  discipline.  Over  time, 
however,  what  has  happened  is  much  better 
than  merely  having  the  different  cultures 
understand  one  another.  Over  time,  many  of 
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the  participants  in  the  planning  process  came 
to  believe  that  it  was  a  waste  of  energy  to 
devote  a  great  deal  of  time  to  translating 
technical  programs  into  various  non-native 
vocabularies.  (Anyone  who  has  engaged  in 
international  negotiations  using  a  translator 
will  readily  acknowledge  this  feeling  of  frus¬ 
tration.)  In  a  mostly  spontaneous  way,  there 
was  progress  towards  the  goal  of  having  a  sin¬ 
gle  SLAD  description  of  the  survivability 
problem,  a  single  vocabulary  for  discussing 
the  technical  details,  and  a  single  view  of  the 
various  tools,  techniques,  and  methodologies 
used  to  solve  survivability  problems. 

It  would  be  difficult  to  overestimate  the  value 
of  the  technical  synergies  that  have  resulted 
from  the  movement  towards  a  single  SLAD 
culture.  Inherently,  the  aspects  of  the  single¬ 
discipline  cultures,  which  have  become  part  of 
the  SLAD  culture,  are  those  which  are 
clearest,  scientifically  soundest,  and  add  the 
most  value  to  Army  systems.  After  all,  we 
are  considering  hundreds  of  SLAD  scientists 
and  engineers  arguing  about  these  matters; 
the  best  ideas  are  the  survivors. 

The  global  synergy  resulting  from  develop¬ 
ment  of  a  more  unified  culture  yields  dozens 
of  specific  technical  synergies  with  respect  to 
particular  problems.  Fault  trees  used  to 
characterize  system  capability  for  one  discip¬ 
line  are  seen  to  be  useful  for  another. 
Software  for  developing  solid-geometry  target 
descriptions  are  seen  as  more  generally  useful 
than  originally  believed.  Comparable  metrics 
of  system  capability  are  developed  in  each 
SLAD  discipline.  This  gives  benefits  meas¬ 
ured  in  both  increased  efficiency  and  in 
improved  quality  survivability  products. 

In  addition  to  the  efficiencies  obtained 
through  the  technical  synergies  mentioned, 
the  formation  of  SLAD  also  caused  efficiencies 
of  a  different  kind.  As  SLAD’s  planning 
teams  developed  and  prioritized  work  pack¬ 
ages,  some  types  of  work  packages  were  con¬ 
sistently  accorded  relatively  low  priority,  and 
consequently  fell  below  the  funding  cut  line. 
This  situation  permitted  us  to  realize  addi¬ 
tional  operating  efficiencies  in  two  ways;  we 
went  out  of  business  areas  which  were  lower 
priority  and  we  were  able  to  reprogram  the 
labor  resources  saved  into  higher  value-added 


activity  where  there  were  more  high  priority 
work  packages  than  personnel  to  execute 
them. 

For  example,  our  EWD  operated  an  airborne 
platform  (BIG  CROW)  out  of  Albuquerque, 
New  Mexico.  This  was  an  expensive  asset  to 
maintain,  was  useful  for  evaluation  of  rela¬ 
tively  few  Army  systems,  and  was  often  used 
in  rote  testing  rather  than  sophisticated 
scientific  experimentation.  Thus  this  asset 
was  transferred  to  TECOM  which  conducts 
routine  technical  testing  for  the  Army.  The 
result  was  not  only  greater  efficiency  for 
SLAD  but  for  the  Army’s  RDA  process. 

As  a  second  example,  our  conventional  ballis¬ 
tics  division  had  a  substantial  team  of  people 
engaged  in  very  high  resolution  force-level 
modeling  for  illumination  of  survivability 
issues.  Many  of  the  work  packages  prepared 
by  this  team  did  not  sift  out  as  high  enough 
priority  to  be  funded.  We  therefore  made  the 
decision  to  terminate  our  efforts  in  that  busi¬ 
ness  area  and  reprogram  the  labor  into 
improving  the  utilization  of  SLAD  products 
within  the  Army’s  existing  force-level  com¬ 
munity.  As  in  the  first  example,  the  result 
was  improved  efficiency  not  only  in  SLAD  but 
for  the  entire  RDA  community. 

In  addition  to  technical  efficiencies  of  the  kind 
discussed,  there  have  been  significant  admin¬ 
istrative  efficiencies  realized  with  the  forma¬ 
tion  of  SLAD.  For  example,  many  Test 
Integration  Working  Groups,  Internal  Process 
Reviews,  quarterly  reviews,  and  other  meet¬ 
ings  about  specific  Army  systems  were  form¬ 
erly  attended  by  each  of  the  five-seven  organi¬ 
zations  which  was  involved  in  survivability 
for  the  Army.  Now,  such  meetings  can  be 
adequately  handled  by  one-two  people,  a  sub¬ 
stantial  improvement. 

This  account  of  the  benefits  of  SLAD’s  organ¬ 
ization  and  disciplined  planning  process 
would  be  incomplete  without  an  assessment  of 
customer  response.  Recall  that  SLAD’s  three 
classes  of  customers  are  Army  PEOs/PMs, 
Acquisition  decision  makers,  and  the  Army’s 
Independent  Technical  Evaluator.  Army  PMs 
have  been  overwhelmingly  satisfied  with 
SLAD’s  one-stop-shopping  concept.  Instead 
of  engaging  in  a  half-dozen  or  more  negotia- 
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tions  on  each  separate  aspect  of  survivability, 
PMs  are  able  to  negotiate  their  entire  sur¬ 
vivability  program  at  once,  with  an  upfront 
guarantee  that  they  will  not  be  blindsided. 
Indeed,  PM  enthusiasm  about  SLAD  help  is 
such  that  SLAD  is  now  a  TIWG  member  and 
a  Test  Evaluation  Master  Plan  signatory  for 
every  major  program.  I  note  that  PM  satis¬ 
faction  with  SLAD  is  perfectly  consistent  with 
the  Defense  Department  initiative  to  reduce 
the  need  for  PM  compliance  with  arbitrary 
standards  and  specifications.  The  world-wide 
threat  is  real  and  evolving,  as  all  PMs  know. 
Moreover,  the  direction  of  that  evolution 
tends  to  be  in  the  direction  of  U.S.  materiel 
weakness.  Neither  PM  shops  nor  their  sup¬ 
porting  Research,  Development,  and 
Engineering  Centers  are  staffed  with  the  kind 
of  multidisciplinary  survivability  expertise 
which  permits  SLAD  to  offer  one-stop  shop¬ 
ping.  For  these  reasons,  Army  PMs  have 
eagerly  continued  to  seek  SLAD  advice  even 
as  the  community  moves  away  from  arbitrary 
standards  and  specifications. 

The  Army’s  Independent  Evaluator  (AMSAA) 
has  been  very  pleased  at  SLAD’s  responsive¬ 
ness  to  its  specific  data  requirements.  Prior 
to  SLAD,  AMSAA  scavenged  for  survivability 
data  in  a  hit-or-miss  fashion.  Sometimes 
AMSAA  induced  PMs  to  pay  for  needed  sur¬ 
vivability  data,  and  sometimes  AMSAA 
employees  were  forced  to  make  crude  guesses 
to  the  detriment  of  the  RDA  process.  SLAD 
took  the  initiative  to  define  AMSAA  vulnera¬ 
bility  requirements  as  part  of  its  Army  mis¬ 
sion.  Since  that  initiative  AMSAA  studies 
have  rested  on  a  more  consistently  credible 
physical  basis  than  before,  and  the  AMSAA 
people  who  conduct  such  studies  have  been 
genuinely  appreciative. 

Acquisition  decision  makers  have  also  been 
well-served  by  the  formation  of  SLAD.  They 
participate  directly  in  the  development,  prior¬ 
itization,  and  review  of  SLAD’s  program; 
they  also  reap  administrative  benefit  from  the 
one-stop-shopping  concept,  especially  in  the 
hectic  period  immediately  preceding  Mile¬ 
stone  Reviews  and  other  significant  decision 
points.  Because  there  is  only  one  organiza¬ 
tion  for  SLA,  the  external  coordination 
requirements  are  greatly  simplified. 


One  way  in  which  the  Army  leadership’s  has 
shown  its  satisfaction  with  SLAD’s  reen¬ 
gineering  of  the  Acquisition  Process,  is  in  giv¬ 
ing  SLAD  new  missions  and  new  resources  to 
subject  to  its  disciplined  planning  process. 
Since  the  formation  of  SLAD  we  have  been 
given  significant  new  responsibilities  in  the 
areas  of:  system  capability  as  a  function  of 
the  natural  environment;  electromagnetic 
environmental  effects  (E3);  and  Soldier  Sur¬ 
vivability.  So  it  is  clear  that  decision  makers 
prefer  SLAD’s  disciplined  planning  approach 
to  analysis. 

ORGANIZATIONAL  BARRIERS 

SLAD’s  reengineering  success  did  not  take 
place  without  the  stimulus  of  various  organi¬ 
zational  barriers.  Since  I  doubt  that  these 
barriers  are  unique  to  SLAD,  I  believe  a  few 
of  them  merit  a  brief  discussion. 

One  barrier  we  faced  in  creating  SLAD  was 
geography.  When  SLAD  was  first  formed  it 
had  significant  activity  at  seven  locations  in 
four  different  states.  We  are  now  down  to  five 
locations  in  three  states,  but  the  fact  remains 
that  our  sites  are  separated  by  thousands  of 
miles.  One  might  argue  that  this  should  be 
irrelevant  in  the  age  of  the  ethernet  and  the 
teleconference  (not  to  mention  the  telephone 
and  the  airplane),  but  our  experience  is  that 
it  is  critical.  If  a  shared  culture  were  already 
in  place  then  the  technological  solutions 
might  suffice.  In  SLAD,  however,  it  was  clear 
that  a  new  culture  had  to  be  built.  This  pro¬ 
cess  proceeded  much  more  slowly  than  would 
have  been  the  case  if  all  of  SLAD  were  co¬ 
located  in  one  place. 

No  single  silver  bullet  enabled  us  to  overcome 
this  problem.  We  had  offsites  for  senior 
managers  and  offsites  including  junior  people. 
We  chartered  Process  Action  Teams  (PATs) 
and  various  working  groups  which  necessi¬ 
tated  face-to-face  meetings.  The  Director  and 
Deputy  Director  each  separately  visited  every 
cost  center  (22)  in  the  organization  to  explain 
our  vision.  We  enforced  the  planning  process 
which  involved  multi-disciplinary  teams. 
And  finally,  we  encouraged  Division  to  Divi¬ 
sion  coordination  and  integration  in  every 
way  possible.  Much  of  the  difficulty  in  cul¬ 
ture  building  was  garden  variety  worker  resis- 
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tance  to  change,  but  this  resistance  was  much 
harder  to  overcome  due  to  SLAD’s  geographi¬ 
cal  separation. 

A  final  barrier  we  had  to  overcome  was  resis¬ 
tance  to  delegated  authority  from  those  to 
whom  we  were  attempting  to  delegate.  As 
described  above,  SLAD’s  planning  process 
forces  fundamental  prioritization  decisions  to 
the  lowest  possible  level  in  the  organization. 
In  the  SLAD  concept  of  operations,  Senior 
managers  review  these  decisions;  they  do  not 
make  them.  This  concept  of  operations  is 
empowering.  It  is  good  Total  Quality 
Management  (TQM).  But  it  was  resisted,  at 
first,  by  many  of  those  we  were  trying  to 
empower.  I  personally  found  this  surprising. 
Some  employees  argued  that  senior  manage¬ 
ment  had  abdicated  its  responsibility;  others 
felt  that  senior  management  was  trying  to 
trick  them  into  making  a  mistake. 

There  is  no  silver  bullet  for  this  barrier 
either.  It  took  time  for  the  workforce  to 
properly  apprehend  that  it  really  was  trusted 
and  empowered.  On  many  occasions  I  had  to 
quite  explicitly  refuse  to  make  decisions  which 
subordinates  asked  me  to  make  while  making 
the  case  that  a  lower  organizational  level  for 
the  decision  was  appropriate.  Eventually,  the 
workforce  came  to  appreciate  that  they  truly 
were  empowered. 

GENERAL  LESSONS  LEARNED 

I  will  highlight  two  pieces  of  general  advice 
which  have  been  touched  on  above  and  which 
seen*  to  have  more  general  applicability  as  we 
in  the  Defense  Department  continue  to  reen¬ 
gineer  the  acquisition  process.  The  first  has 
to  do  with  the  synergies  inherent  in  multidis¬ 
ciplinary  forms  of  organization;  the  second 
with  traditional  distinctions  between  "white 
hat"  and  "black  hat"  organizations. 

In  the  discussion  enumerating  the  technical 
synergies  resulting  from  the  formation  of 
SLAD,  I  noted  that  many  of  the  synergies 
were  a  direct  result  of  putting  several 
scientific  and  engineering  disciplines  together. 
I  believe  that  multidisciplinary  structures  will 
be  the  wave  of  the  future  as  we  continue  to 
reengineer.  We  must  avoid  single-discipline 
stovepiping!  Organizations  which  employ,  for 


example,  solely  atmospheric  physics  or  solely 
operations  research  analysis  will  be  inherently 
less  disciplined,  less  efficient,  and  less  bal¬ 
anced  than  multidisciplinary  organizations 
like  SLAD.  Stovepipe  organizations  are  less 
disciplined  because  they  are  not  intrinsically 
forced  to  consider  which  skill  sets  should  be 
brought  to  bear  on  a  given  problem.  They 
are  less  efficient,  because  they  are  inherently 
unable  to  realize  the  technical  and  adminis¬ 
trative  synergies  that  can  occur  in  an  organi¬ 
zation  with  multiple  areas  of  expertise. 
Finally,  stovepipe  organizations  are  less  bal¬ 
anced  because  to  their  hammer,  every  prob¬ 
lem  appears  as  a  nail. 

My  final  generic  lesson  is  that  we  in  the 
Defense  Department  can  no  longer  afford  the 
luxury  of  separate  "white  hat"  and  black 
hat"  organizations.  Moreover,  to  have  some 
organizations  which  help  PMs  and  others 
which  judge  PMs  is  not  only  expensive,  it  flies 
in  the  face  of  currently  ascendant  manage¬ 
ment  theory. 

Before  Defense  Downsizing,  PMs  had  to  pay 
for  survivability  help;  the  Army  staff  also  had 
to  provide  mission  funds  for  survivability 
assessments.  Now,  increased  recognition  that 
separate  helpers  and  assessors  are  not 
affordable  has  led  to  substantive  dollar  sav¬ 
ings  for  the  DoD.  An  ironic  consequence  is 
that  while  SLAD  is  hard  at  work  playing  both 
roles  with  the  same  resources,  the  two  sets  of 
proponents  are  each  suggesting  that  the  other 
should  pay  the  bill.  One  undesirable  conse¬ 
quence  of  this  situation  is  that  SLAD  senior 
management  has  been  required  to  spend  an 
inordinate  amount  of  time  on  funding  issues. 
Over  time,  this  has  led  to  strong  support  for 
both  our  missions  at  HQDA. 

Current  management  theories  of  quality  indi¬ 
cate  that  if  we  have  flawed  output  from  a 
business  process  we  should  fix  the  process,  not 
hire  additional  checkers  to  monitor  the  flawed 
output.  In  SLAD,  we  saw  above  that  the  goal 
is  to  help  PMs  throughout  the  acquisition 
process  while  simultaneously  feeding  objective 
assessments  into  the  RDA  decision  process  as 
needed.  This  is  simply  a  better  way  to  reen¬ 
gineer  the  process  than  providing  a  lot  of 
black  hat  checkers;  SLAD’s  success  proves 
that  point. 
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